Homebrew

Courtesy of https://twitter.com/MacHomebrew/

Homebrew, the OS X package manager, not the beverage, is the third package manager I've used on OS X. I started in 2005 with Fink and MacPorts. MacPorts seemed to be more active, especially with the flurry of Leopard releases. Indeed it also has some advanced features including dependencies to let you install MAMP stacks. However, I find I prefer to have control of my MAMP stack especially in my volatile development environments, upgrades can be particularly painful with MacPorts.

Homebrew has quite a different approach. There is no need for sudo, all binaries get symlinks in /usr/local, but point to /usr/local/Cellar which controls the formula (aka package) version. The formula scripts are ruby based, so adding your own is very easy.

If you don't want to search forumlae via the command-line then head to http://braumeister.org/ (another GitHub project).

Installation is pretty simple. If you are a developer, e.g. you have Xcode installed it is probably a one-liner. Installation alongside MacPorts is not recommended, but to check you can get everything you need from Homebrew it will probably not cause too many problems.

Now you have brew in your path lets explore some use cases:

• brew list - list installed formulae
• brew search wget - search for formula called wget
• brew info wget - get information about the formula wget
• brew install wget - install wget
• brew uninstall wget -uninstall wget
• brew update - update homebrew (effectively a git pull)
• brew outdated - list formulae which are out of date
• brew upgrade - upgrade all outdated formulae

For more features see the FAQ and follow @MacHomebrew.

Now I wonder if I can convince puppet to use Homebrew over MacPorts...

Operating systems which are current for now

After switching employer I'm no longer tied to Windows Vista & Red Hat. Truth be told I'm quite happy on Red Hat, Vista less so. Anyway, my new setup is Mac OS X Lion & Ubuntu. I use Lion on my personal computer so that is no problem. Although from a developer perspective OS X isn't Linux like Red Hat, but once you learn the tweaks it does play ball.

As for my new Linux workstation I was encouraged to use Ubuntu. I tried Ubuntu briefly at University and didn't really see what the fuss was about. If anything I found it really annoying as everything I wanted (developer/sysadmin tools) were not installed. However, a new job means an open mind and I've landed with Ubuntu 11.10. I had heard about Unity and indeed it is an impressive desktop, which has been a pleasure to use. Ubuntu 12.04 is out in a matter of weeks and will feature HUD, which again is very nice, but is nothing new over one of my favorite Mac apps: Alfred (Like QuickSilver).

Mountain Lion has just been announced as well. So there will be plenty of activity for my operating systems this year, just after I've got them both setup the way I want them!

iOS apps for Science

I was going to list interesting iPhone/iPad apps months ago, but there is already an excellent list available here and I see no point in reinventing the wheel.

If you fancy hunting through the App Store, which isn't that great a search you'll generally find science apps under Books, Education, Reference or Utilities.

Now I have slightly more free time I might have a dabble with the iOS SDK.

Condor 7.2.0 success

Condor 7.2.0 is successfully running on our condor pool. Using our own compiles for Mac/PPC, openSUSE 10.3 64-bit and openSUSE 10.3 32-bit (used on 10.2/10.3/11.0/11.1 computers). Mac/Intel binaries from the condor development team.

For reference the configure options I've used are:

Mac/PPC
./configure --without-classads --without-openssl

openSUSE 10.x/11.0
./configure --disable-glibc-version-check --disable-gcc-version-check --disable-full-port --without-classads --without-gsoap

openSUSE 11.1
./configure --disable-glibc-version-check --disable-gcc-version-check --disable-full-port --without-classads --without-gsoap --without-coredumper

Otherwise compile as per my previous posts.

Make Java look native on Mac

Java is excellent as it just works on so many platforms. My Java software is used on Windows, Linux & Mac. Mac users will notice that the behaviour of Java applications doesn't follow that of the native applications (unlike on Windows/Linux). This is a bit annoying to be honest, however, there are several steps you can take to give a more Mac "feel" to your application.

This link was recently forwarded to me, some of it is certainly useful, although some outdated, but it will give you an idea of what you can alter to add that Mac feel.

MAMP on Leopard part III

The final part to MAMP is PHP. PHP is already installed but not configured. 

Apache needs to be told how to handle .php files (otherwise they are shown as plain text source in the browser). This handler needs to be added to /etc/apache2/httpd.conf, add near similar commands towards the end of the file.

AddType application/x-httpd-php .php

Also check whether the PHP module line is not commented, remove a # if present.

LoadModule php5_module libexec/apache2/libphp5.so

The PHP config file is not active by default, you will want to do this:

sudo cp /etc/php.ini.default /etc/php.ini

As you have installed MySQL, you will want PHP to be able to talk to it you need to edit two lines to ensure it actually works:

mysql.default_socket = /tmp/mysql.sock
mysqli.default_socket = /tmp/mysql.sock

(Add the bold values) Then restart apache for the changes to take affect.

That completes this 3 part installation guide for MAMP on Leopard. P can also be Perl or Python, if you need those languages.

MAMP on Leopard part II

MySQL is the hugely popular SQL database server. It is the only part of MAMP that isn't installed by default with Leopard. No problem, a quick visit to mysql.com will let you download it, select Mac OS X package format. (N.B. Pick the 32-bit version, the 64-bit will cause a headache later if you plan to use Perl's MySQL DBI, see my previous post).

Run all 3 applications within the download: mysql-5.1.X-osx10.5-x86.pkg, MySQLStartupItem.pkg and MySQL.prefPane.

MySQL can be started/stopped via the the MySQL panel in System preferences, or via the command-line:

sudo /Library/StartupItems/MySQLCOM/MySQLCOM {start|stop|restart}

Now it is installed you should remove the anonymous login accounts and add a password to the root account. There is plenty of information on securing MySQL available here.

Configure PHP in part III.

MAMP on Leopard part I

In this three part tutorial I'll detail how to take Mac OS X 10.5 Leopard (client version) from no MAMP to MAMP.

MAMP being a twist on the more common LAMP.

• Mac OS X
  
• Apache
  
• MySQL
  
• PHP (or Python or Perl)
  

MAMP is key for using modern web services and software, e.g. phpMyAdmin, MediaWiki, the list is almost endless. Each part of this tutorial will address one element. I assume you have Mac OS X Leopard installed, if not go buy yourself a Mac!

Apache2 comes with Leopard (and is periodically updated too). All you need to do is switch it on. Jump to System Preferences > Sharing, tick "Web sharing" from the list and apache is now running!

Find your configuration file at /etc/apache2/httpd.conf. Files for the default location can be found in /Library/WebServer/Documents and CGI scripts in /Library/WebServer/CGI-Executables. User websites (http://localhost/~username) can be found in the Sites folder of the specified user. Note the location of any of these can be altered using your httpd.conf. I recommend reading up on the manual before editing your .conf files first.

You can also stop/start apache via the command-line using:

sudo apachectl {start|stop|restart}

MySQL will be covered in part II.

Condor 7.2.0

After not much luck with 7.1.x, I've successfully compiled 7.2.0 on Mac/PPC, openSUSE 10.3 64-bit & openSUSE 11.0 32-bit.

All using the following configure options:

./configure --disable-glibc-version-check --disable-gcc-version-check --disable-full-port --without-classads --without-gsoap

All I need to do now is test it on my pool, but looks promising so far!

MySQL & Perl on Leopard

Mac OS X Leopard comes with Apache2 and PHP pre-installed, but no MySQL. This is the missing link in the very useful MAMP setup. The kind people at mysql.com offer it ready packaged for mac users. When picking your version, MacIntel users pick the 32-bit version, unless you want problems installing Perl's DBD::mysql. What is DBD::mysql? the very useful module that allows Perl scripts to access your MySQL databases. If you install the 64-bit version of MySQL you will hit errors when you install DBD::mysql as a 32-bit installation of Perl is not interested in compiling against 64-bit MySQL libraries.

If you need 64-bit MySQL you could install both versions and compile against the 32-bit libraries.

MacPorts on Leopard with a proxy

MacPorts is an excellent tool for Mac users to grab extra software without having to worry about working out the compile options and read endless instructions etc. As of today there are 5253 ports available for everything from apache2 to iphone apps. The beauty of installing anything from MP is it will install any dependancies as well. I've just upgraded one of our servers to Leopard, but MP didn't want to play anymore. Turns out there is a bug in handling the proxy settings. We have to use a proxy, so this is a show stopper. It looks like they will fix for the 1.8 release, but I can't wait!

Thankfully a simple workaround exists, that requires you to edit /etc/sudoers.

sudo visudo (Don't use vim direct on /etc/sudoers)

in the Default specification section add:

Defaults        env_keep += "http_proxy HTTP_PROXY HTTPS_PROXY FTP_PROXY"
Defaults        env_keep += "ALL_PROXY NO_PROXY"

Essentially when you sudo port install apache2 sudo only takes a stripped down version of current environment variables, so even though you have set your all important $http_proxy, macports doesn't get it. Adding those two lines will fix that. Now install software galore!

Shell scripting

I'm quite a big fan of shell scripting. I use scripts to automate many tasks (normally in conjunction with cron). I've always preferred using bash, opposed to (t)csh. Having just stumbled across this 12 year old post I'm glad I chose bash. Hours of troubleshooting have been avoided!

http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/

Compiling Condor 7.1.4

I've blogged previously on the issues with Condor 7.1.1 onwards - essentially openSUSE and Mac/PPC users need to compile themselves. That is fine if you don't get a error in a core component - condor_c++_util.

However, I've found a workaround that seems to work on 7.1.4 (presumably earlier versions as well).

cd src
./build_init
./configure --disable-glibc-version-check --disable-gcc-version-check --disable-full-port --without-classads --without-gsoap
make

This works on openSUSE 11.0 (32-bit). I've not had time to test on 10.x or 64-bit, but it should work. The difference to my previous configure options is not to build gsoap. Strangely this compiles fine as a dependency, but condor_c++_util will fail when linking against it. Obviously this is a show stopper if you use soap services (luckily for me I don't).

As for compiling on Mac/PPC, hopefully it will work, however, I couldn't get past build_init, which I could previously. So probably just my machine playing up.

Install CruiseControl

CruiseControl (CC) is the third wheel in the pragmatic programming approach. The first two being source version control (e.g. SVN) and testing (e.g. JUnit). It is geared towards Java-based projects, but is so popular it is now available for .NET and Ruby. One of the joys of Java means that installation is a breeze as you can grab the binaries and go.

Installation on Mac OS X & Linux

A servlet server is required to run CC, thankfully one is included in the form of Jetty. You can use an existing servlet server, e.g. Tomcat. I did try to use Tomcat and while I did get the web interfaces to run I did lose some functionality, plus the dashboard isn't as mature as the JSP tool, yet.

Essentially unzip cruisecontrol-bin-2.7.3.zip into /cruisecontrol. Run cruisecontrol.sh and CC is running! Very simple, see the live webpages at http://localhost:8080 and http://localhost:8080/dashboard. Make sure ports 8080 and 8000 are allowed on your firewall. (8000 for JMX, see later). If you don't get anything look in the log file, cruisecontrol.log. You can alter the ports in cruisecontrol.sh.

Now CC is up and running lets go through the components. 

1. The build loop, this is the java process which monitors your selected CVS/SVN repositories, if it detects a change (e.g. you commit something) it will checkout the project to a local directory and attempt to compile/test/package it as per your instructions in your build.xml. If you don't have ant scripts for your projects now is the time to write them. The build loop will then report the results, either a pass or fail with details for failed unit tests/compilation errors.
   
2. The reporting loop, originally this just consisted of JSP pages, now dashboard has come along in web 2.0 glory. Dashboard does everything the JSP pages do and more, plus is far easier on the eye. With both tools you can force a build of your project (this uses JMX and this requires an open port, 8000, to communicate on), if you don't want to wait for the next scheduled build. All the XML reports from the build loop are displayed in the two interfaces.
   

To configure your CC instance edit config.xml, it comes with a sample project in. Use it to construct your project settings and read this to see what else is available (e.g. email users upon a failed build, attach artifacts to successful build, e.g. a jar).

Finally you will want to add Subversion functionality to CC by seeing my previous post. The built-in support doesn't seem to actually work.

Install and setup subversion

We use the popular successor of CVS for our version control, subversion. It is ideal to hold our various projects. In addition to excellent command-line tools it has integration with Eclipse through Subclipse or Subversive and web-based repository viewers, like ViewVC.

Mac OS X
Binary installs are now available from here. Alternatively download from MacPorts or Fink.

openSUSE
You can grab subversion from YaST, but you may not be able to get 1.5 (only 1.4 if using 10.x), thankfully installing the source is very straight forward. Get both the subversion source and dependencies (subversion-deps). Decompress both and run ./configure;make;make install as root. An additional quirk we found on 64-bit openSUSE was that neon must be compiled first:

cd neon
./configure --enable-shared
make
make install

Then run ./configure;make;make install in the folder above.

Create a repository
Use the included admin tool:

svnadmin create /srv/svn

Access this repository as file:///srv/svn, e.g. to see contents issue:

svn list file:///srv/svn

Use svn import, svn ci & svn co to import your initial project, check-in changes and checkout a project. Run svn help to get full details on these and other commands. I'd also recommend checking out the svn book.

To access this repository over a network some additional steps are required so I'll address that in another post.

Condor 7.1.1 onwards update

As I have mentioned previously, condor is dropping support for RH9 and Mac/PPC binaries with 7.1.1 onwards. I have used the 7.1.0 RH9 binaries successfully on openSUSE 10.x. Obviously I'd like to use 7.1.1, well 7.1.2 now, but have no choice but to compile from source. That is fine, but unfortunately I've not managed to get 7.1.1 or 7.1.2 to compile on openSUSE 10.3 or 11.0 for either 32-bit or 64-bit. I also tried compiling on my PowerPC Mac, but again to no avail.

I'd like to continue to use the development series, but unless I can get it compile on these platforms, it won't be an option. I may switch to the stable series, 7.0.x, but alas that doesn't work on openSUSE 11.0 (compiles but doesn't run correctly), therefore stopping me from upgrading our 10.x machines. Hopefully a solution will present itself!

Condor 7.1.1 supported ports

The development series for condor is dropping support for several platforms from 7.1.1 onwards:

1. Red Hat 9 (Suitable for openSUSE 10.x)
2. Solaris 5.8
3. Mac OS X PowerPC

RHEL 3 binaries should be fine for any Red Hat system (and presumably CentOS). Solaris 5.8 users can use the 5.9 binaries.

It should be noted they are continuing these ports for the current stable series (7.0.x).

Unfortunately the RHEL 3 binaries do not work on openSUSE 10.x (well they run but give Shadow exceptions if you try to do anything useful - like run a job!). Looks like a case to compile from source...

UPDATE: Condor 7.1.1 has been pulled due to numerous problems, look out for 7.1.2.

Thumbnails and TeX support for MediaWiki on Mac OS X

After you have setup a new MediaWiki installation you will likely want to enable some extra functionality which requires additional software.

First off to create thumbnails of images you need to install ImageMagick, which gives you the incredibly handy convert program.

Second you can add maths support, using TeX, for this you need ocaml, latex and dvipng. Grab the required programs from MacPorts, they are available from Fink as well.

• sudo port install ImageMagick
• sudo port install ocaml
• sudo port install tetex
• sudo port install ghostscript

Texvc will convert your TeX into whatever MediaWiki wants to display (HTML, MathML or PNG), but it needs to know where several programs are. Given you are probably running your webserver as the www user, who has no $PATH settings how do you tell Texvc where to find the files? In an unusal move, hardcode them! Edit <mediawiki>/math/render.ml, prefixing /opt/local/bin to the four commands

let cmd_dvips tmpprefix = "/opt/local/bin/dvips -R -E " ^ tmpprefix ^ ".dvi -f >" ^ tmpprefix ^ ".ps"
let cmd_latex tmpprefix = "/opt/local/bin/latex " ^ tmpprefix ^ ".tex >/dev/null"
let cmd_convert tmpprefix finalpath = "/usr/local/bin/convert -quality 100 -density 120 " ^ tmpprefix ^ ".ps " ^ finalpath ^ " >/dev/null 2>/dev/null"
let cmd_dvipng tmpprefix finalpath = "/opt/local/bin/dvipng -gamma 1.5 -D 120 -T tight --strict " ^ tmpprefix ^ ".dvi -o " ^ finalpath ^ " >/dev/null 2>/dev/null"

Them recompile texvc with make, ocaml will take over here.

Tell ImageMagick where gs is, by editing /opt/local/lib/ImageMagick-X.X.X/config/delegates.xml, where X.X.X is version number. Replaces every "gs" with "/opt/local/bin/gs", only edit "gs" entries, about half a dozen.

Finally tell MediaWiki to use TeX, by editing your LocalSettings.php with $wgUseTeX = true;

Now math support should be good to go. Thank to this compilation of advice for assistance.

So now this wikitext will produce the following:

== Magical latex in action ==
<math>\left \{ \frac{a}{b} \right \} \quad \left \lbrace \frac{a}{b} \right \rbrace</math>

<math>x \implies y</math> an AMS command

<math>f(n) =
\begin{cases}
n/2,  & \mbox{if }n\mbox{ is even} \\
3n+1, & \mbox{if }n\mbox{ is odd}
\end{cases}</math>

== Image thumbnail ==

[[Image:OpenSUSE.png|frame|Full size|center]]
[[Image:OpenSUSE.png|thumb|A thumbnail|center]]

Start condor on boot with Mac OS X

Once you have condor running on your clients you will want it to load by default when booting. The condor distribution includes linux-based startup scripts, however there are none for the mac. Looking through the mailing list there is a suggestion of scripts to use, but they use Panther (10.3) based technologies, not recommended in Tiger (10.4) and not available in Leopard (10.5).

Delving a bit further I found another way to start condor by using cron.

Create a script to start condor: sudo vim /usr/sbin/start_condor

Enter these contents, and customise to your installation:

#!/bin/bash
# Ensure network is all setup
sleep 100

# Ensure condor environment is loaded
source /opt/condor/condor.sh

# Start condor
/opt/condor/sbin/condor_master

Our condor installation is actually stored on an NFS drive, so the 100 second sleep is to ensure the NFS drives have mounted before the rest of this script runs. I handle the path settings for
$CONDOR_CONFIG, $PATH & $MANPATH in a separate script (condor.sh), alternatively you could specifiy $CONDOR_CONFIG in this script.

Tell cron about your script, and that it should be run on boot:
sudo echo "@reboot root /usr/sbin/start_condor" >> /etc/crontab
I have the condor daemons run as root, hence the root user mentioned in this crontab entry.

Test the script by running direct from the command-line first, if it runs then you should have trouble when rebooting.