Federated SVN

Tue, 26 Jun 2007 04:12:44 GMT

One of the many issues we ran into with a distributed development team literally across the globe is sluggish connectivity - it was so painful that a mere check-in will cost you the time to prepare and finish an entire cup of cappuccino - good for the coffee lovers but not for geek heads.

So without further ado, here's how we build our in-house federated SVN system.

1 Set up SVN servers on both locations.

2 Dump the entire SVN repository from the primary US server and load it to the remote China server, this ensures a equal starting point. The following steps should be done on both servers:

3 Create a synchronization user account

4 Set up password-less SSH for the aforementioned user.

5. Create two new directories: svnlocalcommit and svnremotecommit under /tmp

6 Edit the post-commit script of SVN to include the following: echo $REV > /tmp/svnlocalcommit/$REV, for every commit, this will create a new file under the directory /tmp/svnlocalcommit

7 Load the following "svnpush.sh" script onto the server's /usr/bin directory:

##################################################
#!/bin/bash

VERSION="$1"
DUMPFILE=svndump$VERSION

# create a dump file
svnadmin dump [svn repo path] -r $VERSION --incremental > $DUMPFILE

# push it to remote SVN server
scp $DUMPFILE <user>@<server>:/tmp

# load commit on remote SVN server
ssh <user>@<server> /usr/bin/load_commit.sh $DUMPFILE
exit

##################################################

8 Load the following "load_commit.sh" onto /usr/bin directory:

##################################################

#!/bin/bash

DUMPFILE="$1"

svnadmin load [svn repo path] < /tmp/svnremotecommit/$DUMPFILE

##################################################

9 Load SvnAgent program as the synchronization user

10. Kick off SvnAgent, you're all set.

So here's how it works:

1. Someone makes a commit, the post-commit script will create an empty file named with the rev # under /tmp/svnlocalcommit directory and return, this will be very fast since it's done locally.
2. SyncAgent scans the svnlocalcommit directory every 5 seconds, once a new file is found it'll invoke the svnpush script
3. The svnpush script dumps the local commit (also to /tmp/svnlocalcommit folder but with a "svndump" prefix)
4. The svnpush script then scp the local dump file to the remote server (under remote server's /tmp/svnremotecommit directory)
5. Once scp is successful, the script then commits the uploaded file remotely and returns
6. The SyncAgent daemon receives the return status from the svnpush script
7. If the status was successful, the SyncAgent removes the empty file from svnlocalcommit and sleeps until next scan
8. If the status returns failure, the SyncAgent goes to sleep and try again next time.

Not that hard isn't it?

This works well when commits are coming from one way at a time mostly - namely, one teams works actively while the other team isn't.

Ah, want a copy of the SvnAgent source code? Email me here: k e n 0 6 2 4 7 0 0 [ a t ] g m a i l . c o m

Now get back to work you slacker!

Ken's Space: Technology

Federated SVN