Install CPMD

CPMD is massively parallel code. There is little point running it serially as it will literally take forever to do anything useful. I'm providing the instructions I used to compile the parallel version on our HPC facility, therefore they may need altering for your computer.

Some technical details:

• CPMD 3.13.1
• Intel compilers/MKL 10.1
• openSUSE 10.3, 64-bit

cd SOURCE
./mkconfig.sh IFORT-AMD64-MPI > Makefile

# In Makefile alter
FFLAGS = -pc64 -O2 -unroll
LFLAGS = -L/opt/intel/mkl/10.0.3.020/lib/em64t -lmkl_intel_lp64 -lmkl_sequential -lmkl_core
CC = mpicc -cc=icc
FC = mpif90 -fc=ifort -c
LD = mpif90 -fc-ifort -i-static

Run make, hopefully it will compile happily, copy cpmd.x to somewhere on your $PATH.

It would be wise to test your binary. First download cpmd-test.tar.gz from the contrib section of the downloads (tests are not included with the source). Run as mpisub 5x2 cpmd.x inp-1 or similiar depending on your setup. This command would use 5 dual processor computers.

Condor 7.2.0 success

Condor 7.2.0 is successfully running on our condor pool. Using our own compiles for Mac/PPC, openSUSE 10.3 64-bit and openSUSE 10.3 32-bit (used on 10.2/10.3/11.0/11.1 computers). Mac/Intel binaries from the condor development team.

For reference the configure options I've used are:

Mac/PPC
./configure --without-classads --without-openssl

openSUSE 10.x/11.0
./configure --disable-glibc-version-check --disable-gcc-version-check --disable-full-port --without-classads --without-gsoap

openSUSE 11.1
./configure --disable-glibc-version-check --disable-gcc-version-check --disable-full-port --without-classads --without-gsoap --without-coredumper

Otherwise compile as per my previous posts.

Condor 7.2.0

After not much luck with 7.1.x, I've successfully compiled 7.2.0 on Mac/PPC, openSUSE 10.3 64-bit & openSUSE 11.0 32-bit.

All using the following configure options:

./configure --disable-glibc-version-check --disable-gcc-version-check --disable-full-port --without-classads --without-gsoap

All I need to do now is test it on my pool, but looks promising so far!

Shell scripting

I'm quite a big fan of shell scripting. I use scripts to automate many tasks (normally in conjunction with cron). I've always preferred using bash, opposed to (t)csh. Having just stumbled across this 12 year old post I'm glad I chose bash. Hours of troubleshooting have been avoided!

http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/

Compiling Condor 7.1.4

I've blogged previously on the issues with Condor 7.1.1 onwards - essentially openSUSE and Mac/PPC users need to compile themselves. That is fine if you don't get a error in a core component - condor_c++_util.

However, I've found a workaround that seems to work on 7.1.4 (presumably earlier versions as well).

cd src
./build_init
./configure --disable-glibc-version-check --disable-gcc-version-check --disable-full-port --without-classads --without-gsoap
make

This works on openSUSE 11.0 (32-bit). I've not had time to test on 10.x or 64-bit, but it should work. The difference to my previous configure options is not to build gsoap. Strangely this compiles fine as a dependency, but condor_c++_util will fail when linking against it. Obviously this is a show stopper if you use soap services (luckily for me I don't).

As for compiling on Mac/PPC, hopefully it will work, however, I couldn't get past build_init, which I could previously. So probably just my machine playing up.

Install CruiseControl

CruiseControl (CC) is the third wheel in the pragmatic programming approach. The first two being source version control (e.g. SVN) and testing (e.g. JUnit). It is geared towards Java-based projects, but is so popular it is now available for .NET and Ruby. One of the joys of Java means that installation is a breeze as you can grab the binaries and go.

Installation on Mac OS X & Linux

A servlet server is required to run CC, thankfully one is included in the form of Jetty. You can use an existing servlet server, e.g. Tomcat. I did try to use Tomcat and while I did get the web interfaces to run I did lose some functionality, plus the dashboard isn't as mature as the JSP tool, yet.

Essentially unzip cruisecontrol-bin-2.7.3.zip into /cruisecontrol. Run cruisecontrol.sh and CC is running! Very simple, see the live webpages at http://localhost:8080 and http://localhost:8080/dashboard. Make sure ports 8080 and 8000 are allowed on your firewall. (8000 for JMX, see later). If you don't get anything look in the log file, cruisecontrol.log. You can alter the ports in cruisecontrol.sh.

Now CC is up and running lets go through the components. 

1. The build loop, this is the java process which monitors your selected CVS/SVN repositories, if it detects a change (e.g. you commit something) it will checkout the project to a local directory and attempt to compile/test/package it as per your instructions in your build.xml. If you don't have ant scripts for your projects now is the time to write them. The build loop will then report the results, either a pass or fail with details for failed unit tests/compilation errors.
   
2. The reporting loop, originally this just consisted of JSP pages, now dashboard has come along in web 2.0 glory. Dashboard does everything the JSP pages do and more, plus is far easier on the eye. With both tools you can force a build of your project (this uses JMX and this requires an open port, 8000, to communicate on), if you don't want to wait for the next scheduled build. All the XML reports from the build loop are displayed in the two interfaces.
   

To configure your CC instance edit config.xml, it comes with a sample project in. Use it to construct your project settings and read this to see what else is available (e.g. email users upon a failed build, attach artifacts to successful build, e.g. a jar).

Finally you will want to add Subversion functionality to CC by seeing my previous post. The built-in support doesn't seem to actually work.

Install and setup subversion

We use the popular successor of CVS for our version control, subversion. It is ideal to hold our various projects. In addition to excellent command-line tools it has integration with Eclipse through Subclipse or Subversive and web-based repository viewers, like ViewVC.

Mac OS X
Binary installs are now available from here. Alternatively download from MacPorts or Fink.

openSUSE
You can grab subversion from YaST, but you may not be able to get 1.5 (only 1.4 if using 10.x), thankfully installing the source is very straight forward. Get both the subversion source and dependencies (subversion-deps). Decompress both and run ./configure;make;make install as root. An additional quirk we found on 64-bit openSUSE was that neon must be compiled first:

cd neon
./configure --enable-shared
make
make install

Then run ./configure;make;make install in the folder above.

Create a repository
Use the included admin tool:

svnadmin create /srv/svn

Access this repository as file:///srv/svn, e.g. to see contents issue:

svn list file:///srv/svn

Use svn import, svn ci & svn co to import your initial project, check-in changes and checkout a project. Run svn help to get full details on these and other commands. I'd also recommend checking out the svn book.

To access this repository over a network some additional steps are required so I'll address that in another post.

Condor 7.1.1 onwards update

As I have mentioned previously, condor is dropping support for RH9 and Mac/PPC binaries with 7.1.1 onwards. I have used the 7.1.0 RH9 binaries successfully on openSUSE 10.x. Obviously I'd like to use 7.1.1, well 7.1.2 now, but have no choice but to compile from source. That is fine, but unfortunately I've not managed to get 7.1.1 or 7.1.2 to compile on openSUSE 10.3 or 11.0 for either 32-bit or 64-bit. I also tried compiling on my PowerPC Mac, but again to no avail.

I'd like to continue to use the development series, but unless I can get it compile on these platforms, it won't be an option. I may switch to the stable series, 7.0.x, but alas that doesn't work on openSUSE 11.0 (compiles but doesn't run correctly), therefore stopping me from upgrading our 10.x machines. Hopefully a solution will present itself!

Compile GROMACS with Intel compilers

I recently installed GROMACS on our condor pool. Unsurprisingly code that runs so fast is very dependent on the numerical libraries used during compilation. I therefore chose to use Intel MKL and Intel compiler as they perform very well together. Unfortunately the configure file is not suitable for Intel 10.1 users. I am using GROMACS 3.3.3 (although I hear version 4 should be out imminently), on 32-bit openSUSE 10.3.

First edit your configure file, lines 25897 & 25965:

Alter
LIBS="-lmkl $LIBS"
to
LIBS="-lmkl_intel -lmkl_sequential -lmkl_core $LIBS"

Then run:
./configure CC="icc" CPPFLAGS="-I/opt/intel/mkl/10.0.3.020/include" LDFLAGS="-L/opt/intel/mkl/10.0.3.020/lib/32" --with-fft=mkl
make
make install

You may need to alter the version number or location of your MKL files. I recommend you also download the test set to confirm the compilation. I have 4 fails, but closer inspection revealed they were not show stoppers.

Condor 7.1.1 supported ports

The development series for condor is dropping support for several platforms from 7.1.1 onwards:

1. Red Hat 9 (Suitable for openSUSE 10.x)
2. Solaris 5.8
3. Mac OS X PowerPC

RHEL 3 binaries should be fine for any Red Hat system (and presumably CentOS). Solaris 5.8 users can use the 5.9 binaries.

It should be noted they are continuing these ports for the current stable series (7.0.x).

Unfortunately the RHEL 3 binaries do not work on openSUSE 10.x (well they run but give Shadow exceptions if you try to do anything useful - like run a job!). Looks like a case to compile from source...

UPDATE: Condor 7.1.1 has been pulled due to numerous problems, look out for 7.1.2.

Condor on openSUSE 11.0

Having just installed my first openSUSE 11.0 (32-bit) machine I've thrown condor (7.1.0 binary: RH9, x86, dynamic) on to see it runs fine.

As with openSUSE 10.x you need to install compat-libstdc++ first. As root run: zypper in compat-libstdc++ to do this.

Update: Although compiled, it doesn't behave correctly when running jobs, producing lots of shadow exceptions.

Setup network install for openSUSE 11.0

We run openSUSE on all our linux machines. Therefore, the quickest and easier way to install on lots of machines and ensure quick access to updates is to maintain a local copy of the core repositories onsite.

As we have the entire installation repository we only need network install discs which contain a setup program and then you select whatever packages you want from the repository.

First, download the repositories. There are 3 core repository you need:

1. Installation
2. 3rd party add on software
3. Updates

Find a local mirror and use wget to grab the repositories. The mirror I use in the UK has rsync support, which is very handy for keeping the repositories up to date by only downloading content that has altered.

rsync -Pvptrl --delete rsync://rsync.mirrorservice.org/sites/ftp.opensuse.org/pub/opensuse/distribution/11.0/repo/oss/ /www/suse/SUSE11.0-INSTALL/

rsync -Pvptrl --delete rsync://rsync.mirrorservice.org/sites/ftp.opensuse.org/pub/opensuse/distribution/11.0/repo/non-oss/ /www/suse/SUSE11.0-ADDON/

rsync -Pvptrl --delete rsync://rsync.mirrorservice.org/sites/ftp.opensuse.org/pub/opensuse/update/11.0/ /www/suse/SUSE11.0-UPDATE/

The installation and addon repositories are static, but updates will need updating. Let cron take care of that for you by running the command above once a day.

Next grab the network install CD's, http://download.opensuse.org/distribution/11.0/iso/cd (Don't forget to use a mirror).

We make the repositories available via a local apache web server. So when running the setup program just point to the IP and folder on the web server. During the installation add the addon and update repos, and hey presto fast install with up to date repositories!

Compile condor 7.1.0 on openSUSE 10.3

Condor has traditionally only been available as binaries, most of the time this is fine. We successfully run the Mac OS X PowerPC & Intel binaries and for openSUSE 10.x use the Red Hat 9/dynamic binaries. However for 64-bit openSUSE the binary (RH5) doesn't really work, it struggles to start up. Since the release of condor 7.x they have included the source, so that seemed a sensible avenue to explore. It does compile, but it is a bit more involved than just configure; make; make install!

These instructions should hold true for 7.0.0, 7.0.1 & 7.0.2 as well.

Grab the source for the latest stable or development release (I personally go with the development release).

First off glance through the README, check you have all the prerequisites. I also found I needed to add termcap, terminfo, ncurses-devel and flex (grab them from yast).

Now lets start configuring:

cd src
./build_init
./configure --disable-gcc-version-check --disable-full-port --without-classads

The configure flags mean:

1. Our gcc version is newer than the built-in checks.
2. No standard universe/checkpointing. Standard for a new OS port not to have these.
3. ClassAds not yet supported (no condor_q -better-analyze).

You will get an error:

configure: error: Condor does NOT know what glibc external to use with glibc-2.6.1

To get around this edit configure.ac with your favourite editor. Around line 2500 add this option for the case statement:

"2.6.1" ) # openSUSE 10.3
including_glibc_ext=NO
;;

Now run build_int and configure again as above.

If that runs you can then execute make. Sit back as this may take a while!

If there is a problem, typically on the externals part view the log which will be indicated. You may need to install something (such as the packages I mentioned earlier).

Compilation is finished when something like this pops up (and no obvious errors)

make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/build/condor-7.1.0/src/condor_examples'

Everything is compiled so now prepare the release:

make release (output to release_dir, dynamically linked with debugging, ready for testing)
make public (output to ../public, add stripped dynamic/static linked binaries and no debugging)

Find your final installation bundle in ../public as condor-7.1.0---dynamic.tar.gz. Unpack and use it as you normally would. condor_version, will reveal your custom compile:

$CondorVersion: 7.1.0 May 21 2008 $
$CondorPlatform: X86_64-LINUX_SuSE_UNKNOWN $

Need help/advice? There is an excellent presentation which covers this as well as the users mailing list which is both active and helpful.

Hopefully in the future condor will have better support for openSUSE so that a full port (standard universe/checkpointing) and ClassAds will be available.

Good luck!

openSUSE 11.0 this week

The new version of openSUSE lands this week. It looks like it will be an excellent release building on the successful 10.2 and 10.3 releases.

Find out more: Sneak peaks, screenshots and wiki.

I download a local copy of the installation repository, which I'll post about later this week.