Ken's profileKen's SpaceBlogLists Tools Help

Blog


    June 26

    Federated SVN

    One of the many issues we ran into with a distributed development team literally across the globe is sluggish connectivity - it was so painful that a mere check-in will cost you the time to prepare and finish an entire cup of cappuccino - good for the coffee lovers but not for geek heads.

    So without further ado, here's how we build our in-house federated SVN system.

    1 Set up SVN servers on both locations.

    2 Dump the entire SVN repository from the primary US server and load it to the remote China server, this ensures a equal starting point. The following steps should be done on both servers:

    3 Create a synchronization user account

    4 Set up password-less SSH for the aforementioned user.

    5. Create two new directories: svnlocalcommit and svnremotecommit under /tmp

    6 Edit the post-commit script of SVN to include the following: echo $REV > /tmp/svnlocalcommit/$REV, for every commit, this will create a new file under the directory /tmp/svnlocalcommit

    7 Load the following "svnpush.sh" script onto the server's /usr/bin directory:

    ##################################################
    #!/bin/bash

    VERSION="$1"
    DUMPFILE=svndump$VERSION

    # create a dump file
    svnadmin dump [svn repo path] -r $VERSION --incremental > $DUMPFILE

    # push it to remote SVN server
    scp $DUMPFILE <user>@<server>:/tmp

    # load commit on remote SVN server
    ssh <user>@<server> /usr/bin/load_commit.sh $DUMPFILE
    exit

    ##################################################

    8 Load the following "load_commit.sh" onto /usr/bin directory:

    ##################################################

    #!/bin/bash

    DUMPFILE="$1"

    svnadmin load [svn repo path] < /tmp/svnremotecommit/$DUMPFILE

    ##################################################

    9 Load SvnAgent program as the synchronization user

    10. Kick off SvnAgent, you're all set.

    So here's how it works:

    1. Someone makes a commit, the post-commit script will create an empty file named with the rev # under /tmp/svnlocalcommit directory and return, this will be very fast since it's done locally.
    2. SyncAgent scans the svnlocalcommit directory every 5 seconds, once a new file is found it'll invoke the svnpush script
    3. The svnpush script dumps the local commit (also to /tmp/svnlocalcommit folder but with a "svndump" prefix)
    4. The svnpush script then scp the local dump file to the remote server (under remote server's /tmp/svnremotecommit directory)
    5. Once scp is successful, the script then commits the uploaded file remotely and returns
    6. The SyncAgent daemon receives the return status from the svnpush script
    7. If the status was successful, the SyncAgent removes the empty file from svnlocalcommit and sleeps until next scan
    8. If the status returns failure, the SyncAgent goes to sleep and try again next time.

    Not that hard isn't it?

    This works well when commits are coming from one way at a time mostly - namely, one teams works actively while the other team isn't.

    Ah, want a copy of the SvnAgent source code? Email me here: k e n 0 6 2 4 7 0 0 [ a t ] g m a i l . c o m

    Now get back to work you slacker!



    Comments (1)

    Please wait...
    Sorry, the comment you entered is too long. Please shorten it.
    You didn't enter anything. Please try again.
    Sorry, we can't add your comment right now. Please try again later.
    To add a comment, you need permission from your parent. Ask for permission
    Your parent has turned off comments.
    Sorry, we can't delete your comment right now. Please try again later.
    You've exceeded the maximum number of comments that can be left in one day. Please try again in 24 hours.
    Your account has had the ability to leave comments disabled because our systems indicate that you may be spamming other users. If you believe that your account has been disabled in error please contact Windows Live support.
    Complete the security check below to finish leaving your comment.
    The characters you type in the security check must match the characters in the picture or audio.

    To add a comment, sign in with your Windows Live ID (if you use Hotmail, Messenger, or Xbox LIVE, you have a Windows Live ID). Sign in


    Don't have a Windows Live ID? Sign up

    No namewrote:

    Hi,Do you have used LCDs, used flat screens and secondhand LCDs? Please go here:www.sstar-hk.com(Southern Stars).We are constantly buying re-usable LCD panels and working for LCD recycling.The re-usable panels go through strictly designed process of categorizing, checking, testing, repairing and refurbishing before they are re-used to make remanufactured LCD displays and TV sets.Due to our recent breakthrough in testing and repairing technology of LCD, we can improve the value for your LCD panels. website:www.sstar-hk.com[dgjdbbhjfbbjgdi]

    Sept. 24

    Trackbacks

    The trackback URL for this entry is:
    http://thebentspoon.spaces.live.com/blog/cns!844890C6A37CADFE!149.trak
    Weblogs that reference this entry
    • None