DjangoCon 2014: Top tips for developing and deploying on AWS

I was at my third DjangoCon earlier this month. As with the previous DjangoCon's it was a lot of fun. For the first time I was speaking, which was a first for me, at a tech conference, traditionally I've spoken at science conferences. My talk was entitled "Top tips for developing and deploying on AWS", granted more to do with AWS than Django. However, deploying your shiny new site is just as important as writing it, it's a rather poor website if no one can view it! With the rise of DevOps there has been a push for developers to deploy as well. You can read more on Full Stack Python, an excellent resource maintained by Matt Makai.

Unlike my previous talks this was the first one to be recorded. I'll be honest I was dreading seeing the recording (as I don't like seeing/hearing myself on video). However, I was pleasantly surprised and actually the talk sounded much better that I thought it went! Most importantly for me it will be useful to improve my presentation technique and with video evidence I can track my progress (hopefully anyway).

You can review the slides and videos below - I hope you find it useful.

Book chapter on the use of open source software in the pharmaceutical industry

During my final year at AstraZeneca I was asked to contribute a chapter to "Open source software in life science research: Practical solutions to common challenges in the pharmaceutical industry and beyond". Given this work would be very hard to publish in JCIM, JMC etc and I have never written a book chapter before it was the perfect opportunity. The chapter was entitled: Design Tracker: an easy to use and flexible hypothesis tracking system to aid project team working. It was coauthored by Martin Harrison, who wrote Design Tracker. The abstract sums up best what it covers:

Design Tracker is a hypothesis tracking system used across all sites and research areas in AstraZeneca by the global chemistry community. It is built on the LAMP (Linux, Apache, MySQL, PHP/ Python) software stack, which started as a single server and has now progressed to a six-server cluster running cutting-edge high availability software and hardware. This chapter describes how a local tool was developed into a global production system.

Design Tracker has been mentioned in a few external presentations before but I believe this is the first firm details about it. We talk about its use and how it came to be a global chemistry tool from a prototype at one site. As the book topic suggests we also cover the open source technologies we used to power it. While LAMP is not new, it is not exactly mainstream in the corporate environment for many pharmaceutical companies. We had to harden our setup to make it suitable for 24/7 use, so in addition to the regulars on LAMP we added Red Hat Cluster Suite, Continuent Tungsten and NGINX. We also took the opportunity to move away from apache/mod_python to apache/mod_wsgi. The end result was a service which is available 24/7 and future proofed compared to our previous solution.

The worlds most dynamic and frequently visited websites are powered by similar technologies so they have clearly proven themselves to be suited for the relatively modest needs of a single pharmaceutical company.

The book is available on Amazon UK & US and probably your favourite book reseller as well (ISBN: 978-1907568978). I hope you enjoy our chapter and the many others interesting topics covered.

DjangoCon US 2012

I was lucky enough to attend DjangoCon earlier this month. It was a great experience, I'm still following up on the new tips and tricks I picked up. 

This was my first purely software focused conference, they are normally scientific. There are some stark differences. Notably I'm not the only person tweeting! Thanks to the wonders of lanyrd.com and slideshare.net I was able to leave DC with all the presentations I attended. Although I took copious notes during the talks it is handy to double check details. Plus it was dual track so I missed half the talks, some of which I would of liked to of seen. 

So scientists, why don't you tweet and share your research more? You have gone to the effort of writing (thus getting approval to disclose your work) and attending a conference - that is the hard part. Uploading to slideshare.net and sharing on twitter and/or LinkedIn is the easy step! By the same token I will follow my own advice too.

Using Gunicorn with PyCharm

The Green Unicorn

When you develop projects with Django it comes with a handy runserver which is a lightweight webserver bundled with Django, see more here. It goes without saying you would never want to deploy any production code with this as it doesn't scale nor is it particularly secure. It does not get much attention from the Django developers, lets face it there are better things to work on. So a few alternatives have popped up. The one which caught my eye is gunicorn. Now gunicorn can be used on preprod or production sites as well, but I stick to development here (what I use for production will feature in another post).

Gunicorn is actually a fork of Unicorn which is designed for use with Ruby. It is a WSGI webserver, WSGI is the way to run webservers. It also features integration with Django so can be a drop in replacement for runserver. Installation is a breeze, pip install gunicorn and add gunicorn to your INSTALLED_APPS in your settings.py - Easy.


From the command-line you might do ./manage.py runserver 192.168.1.1:9000 with the Django runserver. To use Gunicorn simply run ./manage.py run_gunicorn 192.168.1.1:9000. Again straight forward. Within PyCharm you need edit your django run configuration and tick the "run custom command" box. Add run_gunicorn and run that. run_gunicorn is a helper function that gunicorn adds to manage.py. Unfortunately, I found not all gunicorn options can be specified via this, which can be annoying.  To access all gunicorn options you need to run gunicorn direct and feed it a wsgi application, handily django is wsgi ready and from 1.4 includes a wsgi file when you create a new project (look for wsgi.py in your project directory). When using gunicorn direct you do not need to list gunicorn in your INSTALLED_APPS. From your virtualenv run gunicorn -b 192.168.1.1:9000 myproject.wsgi:application, where myproject is your Django project name. You can drop in any gunicorn options you like as well. As for running in PyCharm you need to create a Python run configuration, not Django, and provide the following information:

1. Script - /Users/craig/.virtualenvs/blogenv/bin/gunicorn (your virtualenv of choice)
2. Script parameters: -b 192.168.1.1:9000 myproject.wsgi:application
3. Working directory: /Users/craig/PycharmProjects/MyDjangoApp

There is another reason to use gunicorn directly - you can use it with supervisor.

You may come across the gunicorn_django application as well. That is fine to use if you are not on Django 1.4, otherwise use the gunicorn application direct, as shown above.

There is also devserver from David Cramer. I've not used it personally but the other projects that David works on are top rate, so I imagine this is too.

PyCharm for Django development

I have been dabbling with Django for a few years now. Previously, I always used Eclipse for this. Eclipse by itself doesn't support Python or Django but add PyDev or Aptana Studio and it merrily understands what is going on. This was great when I worked on multiple projects in multiple languages. I now find myself working solely in a Python/Django environment. PyCharm was getting rave reviews from my former work colleagues, but I did so little Django it didn't seem worth making the transition. However, with a new role I thought now was the time and I haven't looked back. If you have ever used IntelliJ IDEA for Java you'll be familiar with PyCharm as it is built upon the same framework. Being written in Java also means you are able to code on your Mac, Linux and Windows devices.

For once PyCharm is fully aware of Django, not just syntax highlighting (which was also hit and miss especially in templates mixing HTML/CSS/Django template syntax in Eclipse).  Truth be told I have barely touched the surface of features that PyCharm packs in. See http://www.jetbrains.com/pycharm/features/index.html for a full run down. Each release, of which updates are frequent, either refine or add more features. In the 2.5 release virtualenv support was added, it worked with manual intervention before, but now PyCharm automatically detects them. In addition the support at JetBrains is superb. Admittedly this is a commercial product, although they license favorably for open source project - free.

Just like IntelliJ various plugins are available. A whole host ship by default, Git, CoffeeScript and JavaScript to name a few. More are available online at the plugin repository. The nginx support and NodeJS plugins caught my eye.

One of my favorite features is suport for running mange.py tasks from within the IDE, no need to setup the environment and run from there, as shown (image courtesy of JetBrains).

In fairness I did look at other Python IDEs with Django support, but feel they fall into the same camp as Eclipse. The Django support is just tacked on and is limited. With PyCharm you can see Django support was core to the design and features implemented. I see the PyCharm team will be at DjangoCon US. This is great for two reasons a) I can congratulate them on a great IDE and b) they will be bring a new update (http://blog.jetbrains.com/pycharm/2012/06/pycharm-at-djangocons-around-the-world/). 

Eclipse with Django

Django is popular web framework written in Python. For Python users it is an obvious choice to deploy web applications. Once again I return to my favourite IDE to develop with Django - Eclipse.

Two plugins are needed to bring Django support to Eclipse. The second handles the Django template language. Grab the plugins:

1. PyDev (Python)
2. Aptana Studio (Python + Django)

Both plugins are from Aptana, PyDev is a plugin to their IDE. Thankfully, both work through Eclipse as well.