a Mercurial “super client”
One of the cool trends I’ve seen is the use of distributed version control systems as “super clients” against Subversion. You suck down the entire history of a Subversion repository into a local, private repository, do all of your commits locally, make branches, experiment all you want, then “push” back up to Subversion again. On the internet, nobody knows you’ve been using DVCS (or that you’re a dog.) What’s particularly cool about these bridging tools is that they allow users to try out DVCS before deciding to officially convert a whole project over. Or, if a project happens to be using Subversion but you still want most of the power of a DVCS for local work, it’s a perfect solution.
For all the blabbing I’ve done about distributed version control systems, I’m still a big fan of Mercurial. Of all the DVCSes, I think it’s the easiest to learn for svn users. It has a small, tight set of commands, and the community which runs the project is polite and sane.
In any case, there have been a collection of Mercurial-Subversion bridges available for the last couple of years, but they’ve all been deficient in various ways: either not capturing svn history entirely, or being unable to push back to svn correctly (or only very awkwardly). So I’ve pretty much stayed away. But today I want to plug a new bridge written by a friend of mine (Augie Fackler) who finally did it Right: he wrote a bridge called hgsubversion which (1) uses the actual Subversion API to pull history down (which is faster, more accurate, and long-term sustainable), and (2) actually knows how to push changes back to Subversion correctly. I want the world to be aware of this tool, because I think it’s the first Mercurial-Subversion bridge which deserves to be promoted into the popular ranks with tools like git-svn.
The tool is still young and not generally installable by the public (i.e. you’re not going to find any magic .rpm, .dpkg, .zip or .dmg for it yet)… but here are my cliff notes if you want to start playing with it.
Requirements
- The latest (unreleased) Mercurial
- Local Subversion libraries, at least 1.5, with swig-python bindings built
- A Subversion server that is 1.4 or later
To get the latest Mercurial:
$ hg clone http://selenic.com/repo/hg hg-latest
$ cd hg-latest
$ make
$ sudo make install
To get the latest Subversion python bindings:
$ # if you don't have a binary package for svn-1.5-python-bindings already,
$ # this is a summary of subversion/bindings/swig/INSTALL instructions:
$ svn checkout http://svn.collab.net/repos/svn/tags/1.5.3 svn
$ cd svn
$ ./autogen.sh && ./configure
$ make
$ sudo make install
$ make swig-py # make sure you have swig 1.3 installed already
$ make check-swig-py
$ sudo make install-swig-py
To get hgsubversion:
$ hg clone http://bitbucket.org/durin42/hgsubversion/ ~/hgsubversion
$ cat >> ~/.hgrc
[extensions]
rebase=
svn=/home/user/hgsubversion
^D
To make sure you’re ready to go, do a final sanity check:
$ python -c "import svn.core; print svn.core.SVN_VER_MINOR"
5
$ # if you get something less than 5, you may have conflicting
$ # versions installed, and may need to set PYTHONPATH
Now you can clone your favorite svn repository, and use it locally:
$ hg svnclone http://svn.example.com/repos hg-repos
converting r1 by joe
A trunk/
committed as 24dfb7b51d606a921333e2b8f19a9a6aa5661a69 on branch default
[...]
converting r100 by jane
M trunk/goo
M trunk/moo
committed as 54dfb7b51d6d6a931333e2b8f19a9a6005661a62 on branch default
$
The tool currently assumes a ‘standard’ svn layout of /trunk, /branches, /tags, and then tries to pull them into sane mercurial equivalents. After you’ve made a bunch of local commits, you can push the changes back to subversion:
$ # First merge the latest public svn changes into your repository:
$ hg svn pull
converting r101 by pinky
A trunk/awesomefile
committed as e85afd44dc83d5df2599157096a95b0868de6955 on branch default
$ hg up -C
3 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ # We now have two hg HEADs, because the public svn changes are
$ # considered a different line of development from my own.
$ # For now, rebasing is preferred to merging:
$ hg up -C 9985f017b3ab
3 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ hg svn rebase
saving bundle to /home/user/project/.hg/strip-backup/c4b9dfce6b09-temp
adding branch
adding changesets
adding manifests
adding file changes
added 4 changesets with 4 changes to 4 files
rebase completed
$ # At the moment, each changeset is pushed separately;
$ # changeset flattening not yet implemented
$ hg svn push
Great post!
Hmm, the fact that hgsubversion has to make assumptions about the locations of trunk / tags / branches — assumptions that may not be matched by every repository out there — just makes me think even more that Subversion is going to have to make tags and branches first-class objects somehow, rather than just conventions.
Great post. Gives me something new to try. I also like Karl’s idea of making branches/tags first class objects in Subversion. I guess we’ll see what happens.
Wow, great writeup, Ben. Thanks for plugging this and giving some very useful details on how to actually use it.
Very nice! I think convenient svn integration is a killer feature for “alternative” SCMs.
I have to point out bzr-svn, the official Bazaar-Subversion integration (included by default in the Windows installer too) – it works great and is in heavy use.
Karl: right you are. The branches/tags-as-paths architecture SVN uses is great for people who are new to VCS, but it’s hell for converting to any other VCS. Committing to tags, partial branching, people have all sorts of weird crap going on. It’s flexible, but also makes for a kind of lock-in.
@Karl – it’d be awesome if subversion did treat tags & branches as first class objects. Could it realistically be done?
Thanks for the plug Ben! It’s great to have some notes on use from someone else – I’m sure there are bad assumptions in the code that only work because of my particular workflow.
Obligatory Bazaar plug.
http://bazaar-vcs.org/BzrForeignBranches/Subversion
Bazaar had bzr-svn for a long time. I believe it was the first to offer a transparent, read-write bridge between a DVCS and Subversion.
The current development version is making some great progress towards relaxing requirements on the the consistent use of a specific branching scheme.
Why talking about Mercurial and git, and not at all about Bazaar?
I talk about mercurial because I use it, like it, and want others to try it; I mentioned git only because it’s got the most mindshare of any DVCS at the moment. I left out bzr only because I haven’t tried it yet and don’t know much about it. I also failed to mention darcs, arch, monotone, codeville, and svk. 🙂
I take issue with how your comment could be interpreted as putting bzr in the same league as darcs, arch, monotone cdv and svk.
Git, hg and bzr have much more in common with one another than with the other ones. In terms of both technology and adoption.
Hey, this is great! Everything is working, but I’m not sure exactly what’s going on in the last block of commands. Are the hg svn pull, and then the two updates, necessary? Or can I just skip to the rebase?
Looks good. Might have kept me around longer but I just recently bailed from svn+hg recently. Git+svn lasted a little while but we moved to git completely for the team. Git’s really not much more complex. Local branching and rebasing are miserable to complex in hg. Mq is nice in its simplicity but imo git wins out.
Many of us use mercurial because that’s what the projects we depend on use.
Thank you for your great post!
I’m very interested in completely switching to Mercurial for working on svn projects, and hgsubversion sounds just about right.
Besides: I found your post via the Mercurial wiki.
FYI: This extension can now be used from within MercurialEclipse (r884 and higher).
Now (Dec. 2008), hgsubversion only needs the latest stable release of mercurial (not older than 1.1). No need to check out from hg-latest or crew.
The only problem that remains: the swig-svn bindings (bridging the gap between the svn C-libraries and Python) are not part of the prebuild mercurial binary installations (at least thats the case for win32 binaries). Thus hgsubversion will not work with them for now. You have to build mercurial your self, which turns out to be a bit painfull (again for win32).
Buildinstructions for Mercurial on Windows:
http://www.selenic.com/mercurial/wiki/index.cgi/WindowsInstall
svn-swig-python bindungs:
http://subversion.tigris.org/servlets/ProjectDocumentList?folderID=91&expandFolder=91&folderID=260
Thanks for your great article
I just tested it with the sourceforge.net svn repository for Phex, and for some reason hgsubversion only imports the code since Mar 13 16:55:25 2006.
-> repo: https://phex.svn.sourceforge.net/svnroot/phex/phex
I just wanted to let you know about that.
I can use it anyway for coding, but I can’t use it as a full-fledged mirror of the repository.
Best wishes,
Arne
hgsubversion now works with the current release of Mercurial which includes the rebase extension (hg >=1.1).
Great!
I hope the working won’t stop on this plugin, this is exactly what I miss till now 🙂
Again thanks,
L.
Hey, I’m getting an output I can’t figure out:
$ hg clone svn://10.20.109.140/development/components/webflow
Assuming destination webflow-hg
abort: could not import module encoding!
Any clues?
Nevermind last reply, I just found http://bitbucket.org/durin42/hgsubversion/issue/67/problems-with-encoding-against-crew-hg
Still don’t know how to work around this… is mercurial-1.3 strictly required for using hgsubversion?
I too am stuck on the error message:
abort: could not import module encoding!
I’m trying it with Mercurial 1.1.2 on Ubuntu 9.04 (Jaunty). I have no idea how to progress from here.
Guys, I’m not really a good source of tech support here. You should direct questions to the hgsubversion mailing list: http://groups.google.com/group/hgsubversion
I’m posting this here mostly in hopes of reinforcing Ben’s comment – *please* go to the Google Group with questions/problems. hgsubversion has been growing fairly fast, and the state of things has changed greatly since this blog post. We’re a friendly bunch, so no real need to be wary.
Is it possible to exclude a specific svn directory? Because blender has a gigantic /lib directory with *binary* files (DLLs etc.): https://svn.blender.org/svnroot/bf-blender/trunk/
Panzi, you’re asking your question on a blog post about hgsubversion – I’d recommend going to the google group (http://groups.google.com/group/hgsubversion) and asking there. We’d be more than happy to help you out.
The pre text is really badly formatted… linebreaks were lost.
Hm, the new theme messes up “pre” tags, but “code” tag seems to fix it. All set now. Thanks, Juliano.