The life-changing magic of migrating GitHub repositories with Transposit

I'm not a hoarder. I just happen to manage a ton of GitHub repositories all shoved into one organization. But we need all of these repositories, I swear!

To help developers easily interact with APIs, Transposit has built an assortment of connectors. Each connector lived in its own repository within a single GitHub org. But as our library of connectors grew, our org became more and more cluttered. It was time to KonMari them and tidy up.

The goal#

Create a new GitHub org and migrate our ~100 connector repositories over to it. Do this quickly and comprehensively to minimize disruption to developers interacting with these connectors.

The implementation#

Of course I wanted to automate this  making processes like these straightforward was one of the reasons we built Transposit! But I was still surprised by how simple it was to translate the task into code.

Step 0: Create a new GitHub org#

I did this manually.

Step 1: Fetch the repository names from the existing org#

One of the great features of Transposit is that it takes care of the OAuth dance and credential storage for the developer. After storing a client ID and secret it was just a few clicks to allow Transposit access to my GitHub account.

Step 2: For each connector repository, create its namesake in the new org#

When we were crowding all our repos into one GitHub org, we used the prefix "app_" to distinguish our connector repositories from other code (our main codebase and supporting projects). Thus, I could use this prefix to identify the correct repositories to migrate and then drop it because it was no longer needed.

I made an operation that created the new repo:

My main operation applies it to all repos with the "app_" prefix:

I hit the "Run" button, and off it went.

Step 3: Realize that I've accidentally created all the new repositories as public and run a fix#

Some of our connectors are open source, and some are still a work-in-progress and not quite ready to be public. Whoops. Fortunately, with Transposit it was simple to change them back to private.

Step 4: Migrate repository contents#

I used a python script to copy the repository contents into their new homes. However, while writing this blog post I discovered that GitHub provides an actual repository transfer API I could've used which would have taken care of this step, and combining it with Transposit would've made the process even easier. Welp... good information for next time.

Step 5: Notify my team to update their local repository remotes to point to the new org#

I wrote another script to do this. Again, it would've been better to use the repository transfer API, which will warn you about the updated remote and still push to the new repository.

Step 6: Delete the old repositories#

The end#

There are other ways this migration could've been done. I could've manually transferred ownership of all the repos via the GitHub UI, but that would've taken forever. I could've done the migration using raw requests to the GitHub APIs, but that would've required poring over API and authentication documentation, and would've created more opportunities for error. There are also GitHub-specific CLIs that would have achieved the same outcome, but necessitated learning syntax that I was not likely to use again.

With Transposit, I used JavaScript and SQL to complete the migration in a few hours. The details of my API calls were abstracted away and I got to focus on the logic of my implementation rather than the minutiae of the setup steps. The process was quick and efficient and now all of our repositories are in their proper place.

Your use case might be different, but your sentiment of wanting to build without getting sidetracked in tedium could be the same. Maybe Transposit can spark joy for you too.