Deploying Django

It's been a while between posts, and also between Django projects, but coincidentally with the release of 1.2 I've found an excuse to work with Django again, and have decided to release my own Django deployment library based on fabric called woven. More on that in a minute.

Deployment of Django projects has always been a bit harder than it should be. I understand why Django hasn't gone down the route of providing it's own web server, it just makes a workable deployment strategy critical and time consuming, or limits you to specific hosting solutions like webfaction. The ultimate problem is that since initial deployment is a one off task you tend to cut corners that bite you down the road with updating dependencies and migration. Last time I visited the problem six months ago I ended up settling on Fabric as the simplest and most pythonic way to deploy projects, albeit with a bunch of boilerplate code per project.

With that unsatisfactory state of affairs in mind I cast around for what's new and improved in the deployment landscape. I think it's safe to say that Silver Lining from Ian Bicking is the most significant development in the python deployment world, but at this stage I have to rule it out, as it is kind of a binary decision. Like hosting solutions, either it fits your project or it doesn't. If it doesn't fit there seems a fair bit of work to make it work for you. Another six months and the story might be quite different, since it seems quite clean, and unlike buildout has a much simpler configuration language. The main design decision I disagree with is the omission of fabric & paramiko.

Fabric & paramiko have their warts but are easily the best (and only?) effort to wrap the ssh & bash (or other) shells into a pythonic api. There's not much to learn with fabric because you already know what sudo, run, & cd do, and it's not hard to grasp what most of the other functions do. I'd never used sed before and so it has actually introduced me to new commands. Additionally it's fairly straightforward to handle exceptions and debug when things don't work as expected since you can see the running command output, and stderr can raise a python exception which your script can attempt to handle. The main limitation at the moment is that fabric can't handle a generic remote shell prompt for user input and respond to it (except for password input).

So for the moment I've stuck with my own internal scripts, but have cleaned up and abstracted some of them as woven. Woven currently provides a Django management command setupnode which will setup one or more baseline Ubuntu servers, but the project is obviously more ambitious than that hence the deployment discussion.

Re-use is the key to woven. I use a minimal setup.py, the project's settings.py and Django template api to upload default templates for Apache, Nginx etc. Although there is plenty of configurability, it can be run  by just adding a setup.py file and then woven to installed apps in the project settings file. I'm not really keen on the multiplicity of new ways of configuring things, and unlike some others I'm quite happy with the python module as configuration management. The only issue I have with settings is that I think admin or contrib should have a live settings app to split out startup settings from settings that can be altered without prejudice during runtime and potentially be interoperable between apps. In essence a default common system preferences like os's provide for developers instead of littering settings.py with stuff that probably can live in a database and fixture (but that's another post). Of course at this stage woven doesn't do anything useful to actually deploy your project, so for the moment you'll have to stick with rsync, git/hg pull or whatever you do currently.

Ultimately however I aim to add a four more high level functions, deploy, patch, rollback, and node which need more work, testing and tidying before I can stick them into the repository. The advantage woven aims to offer over a straight rsync or repository pull is that for deploy it will create a single virtual environment for each significant version from setup.py including all dependencies, application media, and webserver configuration. I want to get to the point where I can rollback or forward between versions with minimal dependency drama, and integrate with South or other migration apps for data and database migrations. I also want to be able to not shoot myself in the foot by overwriting deployments, hence once deployed the only way of tinkering with an existing version will be to explicitly patch it, and even then you'll only be able to alter (but not delete) existing files within the project itself. Finally the 4th command node will allow running python manage.py node [managementcommand] against your hosts for adhoc project manipulation.

Finally since woven is just fabric it should be able to be used in a normal fabfile context, making customisation and extension straightforward (though I haven't documented this at this time), or you can make your own custom Django management command based on the WovenCommand base class.

A Github repository for forking/issues/feature requests is here.