cmc/cleberg.net

  1#+date:        [2025-04-05 Sat 23:04:54]
  2#+title:       Automating GitHub to GitLab Repository Sync
  3#+description: Automatically mirroring GitHub repos to GitLab for redundancy.
  4#+slug:        git-mirror
  5#+filetags:    :self-hosting:
  6
  7This is a short post detailing how I maintained repositories on GitHub and
  8mirrors on GitLab - including both public and private repositories.
  9
 10Since GitLab locks pull-only mirrors behind their Premium and Ultimate tiers, I
 11found a different solution.
 12
 13* Creating Mirrors
 14
 15I'll skip the setup and just hit the bullet points:
 16
 17- I have a plethora of GitHub repositories.
 18- I used GitLab's mass-import functionality to do the initial import of my
 19  repositories from GitHub.
 20- I made sure all of my GitHub repositories were cloned locally in my =~/git=
 21  directory.
 22
 23* Setting up Mirror Connections
 24
 25To start, I navigated to the =~/git= directory, which holds all of the
 26repositories I have on GitHub and created a shell script.
 27
 28#+begin_src shell
 29cd ~/git
 30nano setup_mirrors.sh
 31#+end_src
 32
 33Within this shell script, I created a loop that will open each repository and
 34add both the GitHub and GitLab SSH-style clone URIs to the =origin= remote.
 35
 36#+begin_src shell
 37for repo in */
 38do
 39    cd $repo
 40    git remote set-url origin --push --add git@github.com:ccleberg/${repo%*/}.git
 41    git remote set-url origin --push --add git@gitlab.com:ccleberg/${repo%*/}.git
 42    git remote -v
 43    cd ..
 44done
 45#+end_src
 46
 47Once complete, I created another shell script to open each repository, pull any
 48remote commits, and push local commits to both remotes.
 49
 50#+begin_src shell
 51nano mirror.sh
 52#+end_src
 53
 54#+begin_src shell
 55for repo in */;
 56do
 57    cd $repo
 58    git pull --rebase origin HEAD
 59    git push
 60    cd ..
 61done
 62#+end_src
 63
 64Finally, enable execution of both scripts before moving on.
 65
 66#+begin_src shell
 67chmod +x setup_mirrors.sh
 68chmod +x mirror.sh
 69#+end_src
 70
 71* Initialize Mirrors
 72
 73To use the =setup_mirrors.sh= script above, simply execute it from a terminal.
 74
 75#+begin_src shell
 76./setup_mirrors.sh
 77#+end_src
 78
 79Once complete, each repository /should/ have one fetch URI (GitHub) and two push
 80URIs (GitHub & GitLab). At this point, any =git pull= or =git fetch= commands
 81will pull from GitHub and any =git push= commands will send updates to both
 82GitHub and GitLab.
 83
 84* Schedule Periodic Checks
 85
 86To utilize the =mirror.sh= script from the previous step, let's use crontab.
 87
 88#+begin_src shell
 89crontab -e
 90#+end_src
 91
 92Within crontab, I used the schedule below to ensure the script is executed
 93daily.
 94
 95#+begin_src text
 960 0 * * * /Users/cmc/git/mirror.sh
 97#+end_src
 98
 99This ensures that the =mirror.sh= file is executed daily and will push any local
100or GitHub commits to GitLab.
101
102I have tested this by manually running =mirror.sh= and watching the results.
103There are edge cases where I will need to intervene and manually resolve merge
104conflicts, but it's largely autonomous, so I'm happy with the results.