Subversion over a network

I previously talked about creating a subversion repository. But it is only of use if people can access it. 

Using svn list there are several protocols that are accepted:

• file:///Users/svn
  
• http://somedomain.com/svn
  
• svn://mycomputer/Users/svn
• svn+ssh://mycomputer/Users/svn
  

file is used for viewing a repository on your local machine, this approach is not recommended. http is used to view repository served via apache (Binding modules are available for this, see here). svn is the default method, simple but no data encryption. Finally there is svn+ssh, which as the name suggest uses the SSH protocol. This is my method of choice for a group that already uses SSH between workstations.

There are several changes you will need to implement first.

1. Create an svn user and group.
   
2. Change ownership of your repository to the new svn user and group (chown -R svn:svn /Users/svn)
   
3. Change the file permissions to read,write, execute for user and group and nothing for other (chmod -R 770 /Users/svn)
   
4. That change would of just altered an extra permission, reset it manually now: chmod g+s /Users/svn/repo/db
   
5. Add users who can have SSH access to the svn group
   
6. To avoid constantly entering your SSH password create a set of keys.
   

Now you have command-line access sorted! I'd also suggest adding a web interface to view the repository. ViewVC is excellent for this purpose. 

In terms of client applications there are plenty to choose from. Eclipse offers a cross-platform GUI for the repository using either Subclipse or Subversive plug-ins (See my previous post). There are also several other platform specific options: TortoiseSVN (Win), svnX (Mac) and KDEsvn (KDE) come to mind.

Install CruiseControl

CruiseControl (CC) is the third wheel in the pragmatic programming approach. The first two being source version control (e.g. SVN) and testing (e.g. JUnit). It is geared towards Java-based projects, but is so popular it is now available for .NET and Ruby. One of the joys of Java means that installation is a breeze as you can grab the binaries and go.

Installation on Mac OS X & Linux

A servlet server is required to run CC, thankfully one is included in the form of Jetty. You can use an existing servlet server, e.g. Tomcat. I did try to use Tomcat and while I did get the web interfaces to run I did lose some functionality, plus the dashboard isn't as mature as the JSP tool, yet.

Essentially unzip cruisecontrol-bin-2.7.3.zip into /cruisecontrol. Run cruisecontrol.sh and CC is running! Very simple, see the live webpages at http://localhost:8080 and http://localhost:8080/dashboard. Make sure ports 8080 and 8000 are allowed on your firewall. (8000 for JMX, see later). If you don't get anything look in the log file, cruisecontrol.log. You can alter the ports in cruisecontrol.sh.

Now CC is up and running lets go through the components. 

1. The build loop, this is the java process which monitors your selected CVS/SVN repositories, if it detects a change (e.g. you commit something) it will checkout the project to a local directory and attempt to compile/test/package it as per your instructions in your build.xml. If you don't have ant scripts for your projects now is the time to write them. The build loop will then report the results, either a pass or fail with details for failed unit tests/compilation errors.
   
2. The reporting loop, originally this just consisted of JSP pages, now dashboard has come along in web 2.0 glory. Dashboard does everything the JSP pages do and more, plus is far easier on the eye. With both tools you can force a build of your project (this uses JMX and this requires an open port, 8000, to communicate on), if you don't want to wait for the next scheduled build. All the XML reports from the build loop are displayed in the two interfaces.
   

To configure your CC instance edit config.xml, it comes with a sample project in. Use it to construct your project settings and read this to see what else is available (e.g. email users upon a failed build, attach artifacts to successful build, e.g. a jar).

Finally you will want to add Subversion functionality to CC by seeing my previous post. The built-in support doesn't seem to actually work.

Install and setup subversion

We use the popular successor of CVS for our version control, subversion. It is ideal to hold our various projects. In addition to excellent command-line tools it has integration with Eclipse through Subclipse or Subversive and web-based repository viewers, like ViewVC.

Mac OS X
Binary installs are now available from here. Alternatively download from MacPorts or Fink.

openSUSE
You can grab subversion from YaST, but you may not be able to get 1.5 (only 1.4 if using 10.x), thankfully installing the source is very straight forward. Get both the subversion source and dependencies (subversion-deps). Decompress both and run ./configure;make;make install as root. An additional quirk we found on 64-bit openSUSE was that neon must be compiled first:

cd neon
./configure --enable-shared
make
make install

Then run ./configure;make;make install in the folder above.

Create a repository
Use the included admin tool:

svnadmin create /srv/svn

Access this repository as file:///srv/svn, e.g. to see contents issue:

svn list file:///srv/svn

Use svn import, svn ci & svn co to import your initial project, check-in changes and checkout a project. Run svn help to get full details on these and other commands. I'd also recommend checking out the svn book.

To access this repository over a network some additional steps are required so I'll address that in another post.

Condor 7.1.1 onwards update

As I have mentioned previously, condor is dropping support for RH9 and Mac/PPC binaries with 7.1.1 onwards. I have used the 7.1.0 RH9 binaries successfully on openSUSE 10.x. Obviously I'd like to use 7.1.1, well 7.1.2 now, but have no choice but to compile from source. That is fine, but unfortunately I've not managed to get 7.1.1 or 7.1.2 to compile on openSUSE 10.3 or 11.0 for either 32-bit or 64-bit. I also tried compiling on my PowerPC Mac, but again to no avail.

I'd like to continue to use the development series, but unless I can get it compile on these platforms, it won't be an option. I may switch to the stable series, 7.0.x, but alas that doesn't work on openSUSE 11.0 (compiles but doesn't run correctly), therefore stopping me from upgrading our 10.x machines. Hopefully a solution will present itself!

Compile GROMACS with Intel compilers

I recently installed GROMACS on our condor pool. Unsurprisingly code that runs so fast is very dependent on the numerical libraries used during compilation. I therefore chose to use Intel MKL and Intel compiler as they perform very well together. Unfortunately the configure file is not suitable for Intel 10.1 users. I am using GROMACS 3.3.3 (although I hear version 4 should be out imminently), on 32-bit openSUSE 10.3.

First edit your configure file, lines 25897 & 25965:

Alter
LIBS="-lmkl $LIBS"
to
LIBS="-lmkl_intel -lmkl_sequential -lmkl_core $LIBS"

Then run:
./configure CC="icc" CPPFLAGS="-I/opt/intel/mkl/10.0.3.020/include" LDFLAGS="-L/opt/intel/mkl/10.0.3.020/lib/32" --with-fft=mkl
make
make install

You may need to alter the version number or location of your MKL files. I recommend you also download the test set to confirm the compilation. I have 4 fails, but closer inspection revealed they were not show stoppers.