Skip to main content

Git Repository Migration

Introduction​

Git repository migration is the process of moving a Git repository from one hosting platform or server to another while preserving its history, branches, tags, and other metadata. Whether you're switching from GitHub to GitLab, migrating to a self-hosted solution, or consolidating repositories, understanding how to properly migrate Git repositories is an essential skill for developers and administrators.

In this guide, we'll explore various approaches to Git repository migration, common challenges, and best practices to ensure a smooth transition.

Why Migrate Git Repositories?​

There are several reasons why you might need to migrate a Git repository:

  • Switching between hosting services (GitHub, GitLab, Bitbucket, etc.)
  • Moving from cloud-hosted to self-hosted solutions (or vice versa)
  • Consolidating multiple repositories into a single repository
  • Splitting a monolithic repository into smaller ones
  • Company acquisition or reorganization
  • Cost reduction or feature requirements

Prerequisites​

Before proceeding with a Git repository migration, ensure you have:

  • Git installed and configured on your local machine
  • Appropriate access permissions for both source and destination repositories
  • Basic understanding of Git commands and workflows
  • Backup of important data (just in case!)

Basic Repository Migration Techniques​

Method 1: Clone and Push​

The simplest method to migrate a Git repository is to clone it from the source and push it to the destination.

bash
# Step 1: Create a mirror clone of the repository (includes all branches and tags)
git clone --mirror https://source-host.com/user/repo.git

# Step 2: Navigate to the cloned repository
cd repo.git

# Step 3: Add the new remote repository URL
git remote set-url origin https://destination-host.com/user/repo.git

# Step 4: Push all references (branches, tags, etc.) to the new remote
git push --mirror

This approach works well for simple migrations between compatible hosting providers.

Method 2: Using Git Bundle​

For repositories that are too large or when you need an offline migration approach:

bash
# Step 1: On the source machine, create a bundle file
git bundle create repo.bundle --all

# Step 2: Transfer the bundle file to the destination machine

# Step 3: On the destination machine, clone from the bundle
git clone repo.bundle repo
cd repo

# Step 4: Add the new remote and push
git remote add origin https://destination-host.com/user/repo.git
git push --all
git push --tags

Method 3: Using Provider-Specific Import Tools​

Most Git hosting providers offer built-in tools for repository migration:

  • GitHub: Import repository feature
  • GitLab: Import project feature
  • Bitbucket: Import repository feature

These tools often handle additional metadata like pull requests, issues, and wiki pages.

Advanced Migration Scenarios​

Migrating Large Repositories​

For repositories with large files or long history:

bash
# Option 1: Use Git LFS for large files
git lfs install
git lfs track "*.psd" "*.zip" # Track large file extensions
git add .gitattributes
git commit -m "Configure Git LFS"

# Option 2: Use partial clone (Git 2.22+)
git clone --filter=blob:none https://source-host.com/user/repo.git

Preserving Git Hooks and Configuration​

Git hooks and configuration aren't included in standard clones:

bash
# Copy Git hooks manually
cp -r source-repo/.git/hooks/* destination-repo/.git/hooks/

# Transfer specific Git configuration
cp source-repo/.git/config destination-repo/.git/config

Migrating Multiple Repositories​

For migrating multiple repositories, you can automate the process:

bash
#!/bin/bash
repos=("repo1" "repo2" "repo3")
source_host="https://source-host.com/user"
dest_host="https://destination-host.com/user"

for repo in "${repos[@]}"
do
  git clone --mirror "$source_host/$repo.git"
  cd "$repo.git"
  git remote set-url origin "$dest_host/$repo.git"
  git push --mirror
  cd ..
done

Repository Structure Modifications During Migration​

Combining Multiple Repositories​

To merge several repositories while preserving history:

bash
# Create a new repository
mkdir combined-repo && cd combined-repo
git init

# Add multiple remotes
git remote add repo1 ../repo1
git remote add repo2 ../repo2

# Fetch all remotes
git fetch --all

# Merge repositories into subdirectories
git merge repo1/main --allow-unrelated-histories
mkdir repo1-files && mv !(repo1-files) repo1-files
git add .
git commit -m "Move repo1 files to subdirectory"

git merge repo2/main --allow-unrelated-histories
mkdir repo2-files && mv !(repo1-files|repo2-files) repo2-files
git add .
git commit -m "Move repo2 files to subdirectory"

Splitting a Repository​

To extract a subdirectory into a separate repository:

bash
# Extract subdirectory with history
git subtree split -P path/to/subdirectory -b split-branch

# Create new repository from split branch
mkdir new-repo && cd new-repo
git init
git pull ../original-repo split-branch

Migration Workflow Diagram​

Platform-Specific Migration​

GitHub to GitLab Migration​

  1. Create a new empty GitLab repository
  2. Clone the GitHub repository with all references:
bash
git clone --mirror https://github.com/user/repo.git
cd repo.git
git remote add gitlab https://gitlab.com/user/repo.git
git push gitlab --mirror
  1. To migrate issues and pull requests, use GitLab's GitHub importer.

Bitbucket to GitHub Migration​

  1. Create an empty GitHub repository
  2. Migrate the repository:
bash
git clone --mirror https://bitbucket.org/user/repo.git
cd repo.git
git remote set-url origin https://github.com/user/repo.git
git push --mirror
  1. For issues and pull requests, consider using migration tools like GitHub's importer.

Best Practices for Repository Migration​

  1. Plan the migration carefully:

    • Document all repositories to be migrated
    • Create a migration timeline
    • Identify critical repositories that need special attention

  2. Communication:

    • Notify all stakeholders about the migration
    • Provide clear instructions for updating remotes
    • Schedule migration during off-hours if possible

  3. Pre-migration checks:

    • Verify access permissions
    • Check for large files that may cause issues
    • Review branch structures and naming conventions

  4. Post-migration verification:

    • Compare commit hashes between source and destination
    • Verify all branches and tags are present
    • Test the build process with the new repository
    • Ensure CI/CD pipelines are properly updated

  5. Clean up:

    • Archive or mark original repositories as deprecated
    • Update documentation to reflect new repository locations
    • Remove temporary clones and bundles

Troubleshooting Common Issues​

Missing Branches or Tags​

If branches or tags are missing after migration:

bash
# Check if all refs were fetched
git branch -a
git tag -l

# Push specific missing branches
git push origin branch-name

# Push all tags
git push --tags

Authentication Issues​

For authentication problems:

bash
# Use personal access token instead of password
git remote set-url origin https://username:[email protected]/username/repo.git

# Or configure credential helper
git config --global credential.helper store

Large Files Causing Failures​

If repository contains large files causing push failures:

bash
# Identify large files
git rev-list --objects --all | grep -f <(git verify-pack -v .git/objects/pack/*.idx | sort -k 3 -n | tail -10 | awk '{print $1}')

# Consider using Git LFS or removing large files using git-filter-repo

External Tools for Git Migration​

Several tools can assist with complex migrations:

  • git-filter-repo: A powerful tool for rewriting repository history
  • BFG Repo-Cleaner: Designed to remove unwanted data from Git repositories
  • GitHub Importer: Built-in tool for importing repositories to GitHub
  • GitLab Importer: Comprehensive tool for importing projects into GitLab

Summary​

Git repository migration is a common task in modern development workflows. By understanding the different migration approaches and best practices, you can successfully migrate repositories while preserving history and minimizing disruption to your team.

Remember that the appropriate migration method depends on your specific requirements, repository size, and the platforms involved. Always test your migration process with a small repository before attempting large-scale migrations.

Additional Resources​

Exercises​

  1. Practice migrating a small test repository between two different Git hosting providers.
  2. Try using the Git bundle approach to migrate a repository without internet access.
  3. Create a script to automate the migration of multiple repositories.
  4. Experiment with modifying repository structure during migration (e.g., splitting a subdirectory into its own repository).
  5. Compare the results of different migration approaches and document the pros and cons of each method.

If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)