Condor 7.2.0 success

Condor 7.2.0 is successfully running on our condor pool. Using our own compiles for Mac/PPC, openSUSE 10.3 64-bit and openSUSE 10.3 32-bit (used on 10.2/10.3/11.0/11.1 computers). Mac/Intel binaries from the condor development team.

For reference the configure options I've used are:

Mac/PPC
./configure --without-classads --without-openssl

openSUSE 10.x/11.0
./configure --disable-glibc-version-check --disable-gcc-version-check --disable-full-port --without-classads --without-gsoap

openSUSE 11.1
./configure --disable-glibc-version-check --disable-gcc-version-check --disable-full-port --without-classads --without-gsoap --without-coredumper

Otherwise compile as per my previous posts.

Condor 7.2.0

After not much luck with 7.1.x, I've successfully compiled 7.2.0 on Mac/PPC, openSUSE 10.3 64-bit & openSUSE 11.0 32-bit.

All using the following configure options:

./configure --disable-glibc-version-check --disable-gcc-version-check --disable-full-port --without-classads --without-gsoap

All I need to do now is test it on my pool, but looks promising so far!

Compiling Condor 7.1.4

I've blogged previously on the issues with Condor 7.1.1 onwards - essentially openSUSE and Mac/PPC users need to compile themselves. That is fine if you don't get a error in a core component - condor_c++_util.

However, I've found a workaround that seems to work on 7.1.4 (presumably earlier versions as well).

cd src
./build_init
./configure --disable-glibc-version-check --disable-gcc-version-check --disable-full-port --without-classads --without-gsoap
make

This works on openSUSE 11.0 (32-bit). I've not had time to test on 10.x or 64-bit, but it should work. The difference to my previous configure options is not to build gsoap. Strangely this compiles fine as a dependency, but condor_c++_util will fail when linking against it. Obviously this is a show stopper if you use soap services (luckily for me I don't).

As for compiling on Mac/PPC, hopefully it will work, however, I couldn't get past build_init, which I could previously. So probably just my machine playing up.

Condor 7.1.1 onwards update

As I have mentioned previously, condor is dropping support for RH9 and Mac/PPC binaries with 7.1.1 onwards. I have used the 7.1.0 RH9 binaries successfully on openSUSE 10.x. Obviously I'd like to use 7.1.1, well 7.1.2 now, but have no choice but to compile from source. That is fine, but unfortunately I've not managed to get 7.1.1 or 7.1.2 to compile on openSUSE 10.3 or 11.0 for either 32-bit or 64-bit. I also tried compiling on my PowerPC Mac, but again to no avail.

I'd like to continue to use the development series, but unless I can get it compile on these platforms, it won't be an option. I may switch to the stable series, 7.0.x, but alas that doesn't work on openSUSE 11.0 (compiles but doesn't run correctly), therefore stopping me from upgrading our 10.x machines. Hopefully a solution will present itself!

Condor 7.1.1 supported ports

The development series for condor is dropping support for several platforms from 7.1.1 onwards:

1. Red Hat 9 (Suitable for openSUSE 10.x)
2. Solaris 5.8
3. Mac OS X PowerPC

RHEL 3 binaries should be fine for any Red Hat system (and presumably CentOS). Solaris 5.8 users can use the 5.9 binaries.

It should be noted they are continuing these ports for the current stable series (7.0.x).

Unfortunately the RHEL 3 binaries do not work on openSUSE 10.x (well they run but give Shadow exceptions if you try to do anything useful - like run a job!). Looks like a case to compile from source...

UPDATE: Condor 7.1.1 has been pulled due to numerous problems, look out for 7.1.2.

Start condor on boot with Mac OS X

Once you have condor running on your clients you will want it to load by default when booting. The condor distribution includes linux-based startup scripts, however there are none for the mac. Looking through the mailing list there is a suggestion of scripts to use, but they use Panther (10.3) based technologies, not recommended in Tiger (10.4) and not available in Leopard (10.5).

Delving a bit further I found another way to start condor by using cron.

Create a script to start condor: sudo vim /usr/sbin/start_condor

Enter these contents, and customise to your installation:

#!/bin/bash
# Ensure network is all setup
sleep 100

# Ensure condor environment is loaded
source /opt/condor/condor.sh

# Start condor
/opt/condor/sbin/condor_master

Our condor installation is actually stored on an NFS drive, so the 100 second sleep is to ensure the NFS drives have mounted before the rest of this script runs. I handle the path settings for
$CONDOR_CONFIG, $PATH & $MANPATH in a separate script (condor.sh), alternatively you could specifiy $CONDOR_CONFIG in this script.

Tell cron about your script, and that it should be run on boot:
sudo echo "@reboot root /usr/sbin/start_condor" >> /etc/crontab
I have the condor daemons run as root, hence the root user mentioned in this crontab entry.

Test the script by running direct from the command-line first, if it runs then you should have trouble when rebooting.

Condor on openSUSE 11.0

Having just installed my first openSUSE 11.0 (32-bit) machine I've thrown condor (7.1.0 binary: RH9, x86, dynamic) on to see it runs fine.

As with openSUSE 10.x you need to install compat-libstdc++ first. As root run: zypper in compat-libstdc++ to do this.

Update: Although compiled, it doesn't behave correctly when running jobs, producing lots of shadow exceptions.

Compile condor 7.1.0 on openSUSE 10.3

Condor has traditionally only been available as binaries, most of the time this is fine. We successfully run the Mac OS X PowerPC & Intel binaries and for openSUSE 10.x use the Red Hat 9/dynamic binaries. However for 64-bit openSUSE the binary (RH5) doesn't really work, it struggles to start up. Since the release of condor 7.x they have included the source, so that seemed a sensible avenue to explore. It does compile, but it is a bit more involved than just configure; make; make install!

These instructions should hold true for 7.0.0, 7.0.1 & 7.0.2 as well.

Grab the source for the latest stable or development release (I personally go with the development release).

First off glance through the README, check you have all the prerequisites. I also found I needed to add termcap, terminfo, ncurses-devel and flex (grab them from yast).

Now lets start configuring:

cd src
./build_init
./configure --disable-gcc-version-check --disable-full-port --without-classads

The configure flags mean:

1. Our gcc version is newer than the built-in checks.
2. No standard universe/checkpointing. Standard for a new OS port not to have these.
3. ClassAds not yet supported (no condor_q -better-analyze).

You will get an error:

configure: error: Condor does NOT know what glibc external to use with glibc-2.6.1

To get around this edit configure.ac with your favourite editor. Around line 2500 add this option for the case statement:

"2.6.1" ) # openSUSE 10.3
including_glibc_ext=NO
;;

Now run build_int and configure again as above.

If that runs you can then execute make. Sit back as this may take a while!

If there is a problem, typically on the externals part view the log which will be indicated. You may need to install something (such as the packages I mentioned earlier).

Compilation is finished when something like this pops up (and no obvious errors)

make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/build/condor-7.1.0/src/condor_examples'

Everything is compiled so now prepare the release:

make release (output to release_dir, dynamically linked with debugging, ready for testing)
make public (output to ../public, add stripped dynamic/static linked binaries and no debugging)

Find your final installation bundle in ../public as condor-7.1.0---dynamic.tar.gz. Unpack and use it as you normally would. condor_version, will reveal your custom compile:

$CondorVersion: 7.1.0 May 21 2008 $
$CondorPlatform: X86_64-LINUX_SuSE_UNKNOWN $

Need help/advice? There is an excellent presentation which covers this as well as the users mailing list which is both active and helpful.

Hopefully in the future condor will have better support for openSUSE so that a full port (standard universe/checkpointing) and ClassAds will be available.

Good luck!

Condor for number crunching

We have a need for HPC within our group (quantum chemistry calculations, machine learning, molecular dynamics simulations & analysis, etc.). To fulfil this need we have several SGE-based clusters within our department and the university. Our local clusters were in need of a refresh (multiple dated OS's - Red Hat 7!) and ideally needed to be unified somehow. It became tiresome having multiple clusters to pick from. What one has the most free slots, or the shortest queue? If it is full you would have to move all your data and get setup to run on a different cluster. We needed something to maximise our use of the compute nodes but simplify the submission process to avoid wasting time.

We opted for a more grid-based solution: condor. The reasons for this were:

• All our local clusters are now combined into one condor pool.
• It removes the needs for multiple head nodes, as users can submit direct from their desktops.
• Cross-platform so you can use with Windows, Linux & Mac.
• Grid approach means we take advantage of our desktop computers as well.
  

We still use the university's central SGE cluster, it is an invaluable resource. However, condor allows us to make the most of our local resources which are exclusively for our use.

Find out more about condor here: http://www.cs.wisc.edu/condor/. The annual Condor Week now has videos of some of tutorials (as well as slides) so check out what it is all about.

Image courtesy of Wikipedia.