Christoph Rüegg

Maintaining .NET open source projects for 15 years and counting. Enjoying functional programming, distributed algorithms and number crunching.

Twitter GitHub NuGet Keybase.io LinkedIn Paypal

Math.NET

Math.NET is an opensource initiative to build and maintain toolkits covering fundamental mathematics, targeting advanced but also every day needs of .Net developers.

Contributions

FsBlog FsLab Paket

Software Engineering

Sonova Hearing Aid Fitting

Lokad Business Analytics

Microsoft SDC Zurich

ewl energie wasser luzern

Swiss Army Comm/GIS

NagraVision Kudelski Group

Subscribe

GitHub recently started publishing all pull request as special git refs. This is awesome, since it makes it trivial to checkout out and work with them from your local repository, without having to add the submitter's repo as a remote all the time. It is also nicely done in that it does not affect normal clones in any way - unless you actually want to fetch them.

However, there is one case where it may have an undesired side effect: mirrors. For example, I routinely mirror the Math.NET Numerics mainline repository to a couple other places, including Codeplex, Gitorious and Google Code. I want a mirror to exactly mirror the source repository, adding all new branches and tags automatically, but also remove those that have been deleted in the source. Git has excellent support for such exact mirroring. Unfortunately this mirroring mechanism includes all the pull refs as well, which may not be what you want. In Math.NET Numerics, some pull request actually base on an old (long removed) branch that included some corrupt objects. So in this case, including them in the mirror not only doubles the repository size, it also causes a corrupt git file system.

Luckily there is an easy way to skip them in the mirror, but to do that we must understand how git refs actually work:

Git Refs

In essence, a git ref is just a reference to a specific git commit. Refs can represent local branches and tags, but also remote branches. To keep things organized, they're structured hierarchically. You can find them in two places:

As separate file for each ref in the .git/refs directory

In the .git/packed-refs file

In a normal local repository you'll typically end up with the following structure:

refs/heads/{branchname} - all your local branches
refs/remotes/{remotename}/{branchname}` - all your fetched remote branches
refs/tags/{tagname} - all tags
refs/stash - your stash, if you use it
However, if you create a local mirror of a GitHub repo, i.e.
$ git clone --mirror git://github.com/mathnet/mathnet-numerics.git
Then you'll end up with exactly the same bare structure as the remote itself,
but this time including GitHub's pull requests:
refs/heads/{branchname} - all your remote branches
refs/tags/{tagname} - all your remote tags
refs/pull/{id}/head|merge - all your remote GitHub pull requests
How exactly a remote's refs are mapped down to your local refs and why there
is a difference between the structure of a normal clone and a bare mirror
is defined in the refspec that is automatically added to your repo config.
In a normal clone, the fetch refspec typically looks like this:
fetch = +refs/heads/*:refs/remotes/origin/*
It essentially says that all remote refs within refs/heads should map
to local refs in refs/remotes/origin. On the other hand, a mirror
includes all refs, so its refspec looks like the following:
fetch = +refs/*:refs/*
Excluding Pull Refs
As far as I know there is no simple way to exclude some refs in a subdirectory
of a refspec, but you can add multiple fetch refspecs to get the same effect.
Simply replace the catch-all refspec above with two more specific specs to just
include all heads and tags, but not the pulls, and all the remote pull refs
will no longer make it into your bare mirror:
fetch = +refs/heads/*:refs/heads/*
fetch = +refs/tags/*:refs/tags/*
Full Config Example
For completeness, I've attached the full config (see git config -e)
I use myself for mirroring Math.NET Numerics below.
[core]
    repositoryformatversion = 0
    filemode = false
    bare = true
    symlinks = false
    ignorecase = true
    hideDotFiles = dotGitOnly
[remote "mathnet"]
    url = [email protected]:mathnet/mathnet-numerics.git
    fetch = +refs/heads/*:refs/heads/*
    fetch = +refs/tags/*:refs/tags/*
    mirror = true
[remote "mirrors"]
    url = https://git01.codeplex.com/mathnetnumerics
    url = https://code.google.com/p/mathnet-numerics/
    url = [email protected]:mathnet-numerics/mainline.git
    mirror = true
    skipDefaultUpdate = true
To update the mirror, I then run the following commands:
$ git remote update
$ git push mirrors