Friday, 29 August 2008

4. A Django-Powered Weblog

Creating the Weblog Application

This is part 4 of a series of posts on James Bennett's excellent Practical Django Projects. The table of contents and explanation can be found here. Before we start going through the Weblog app, you may want to have a look at my post on Django tests. This will help sort out any basic errors you make typing up examples.

In this section of the book James gives some useful tips for organising your project files, which are worth taking on board. I also recommend checking out Eric Florenzano's screencast on the topic. Once you get to the point of needing to maintain different combinations of versions of packages you'll want something more like zc.buildout or virtualenv but for now symlinks or altering your PYTHONPATH is going to be fine.

Designing the Models

On p47 we design the Category model. As usual remove the Admin class and create your own admin.py. If you want to do tests as well go ahead and create tests.py as well
#admin.py
from django.contrib import admin
from coltrane.models import Category

admin.site.register(Category)
#tests.py
from coltrane import admin, models, views
Add coltrane to the installed apps as described on p48 and then run python manage.py test to ensure you haven't made any typos. Run python manage.py syncdb and continue with p48.

On p50 instead of adding prepopulate_from=['title'] to your Category model reorganise your admin.py instead to create a custom admin class.
#admin.py
from django.contrib import admin
from coltrane.models import Category

class CategoryAdmin(admin.ModelAdmin):
 prepopulated_fields = {"slug":("title",)}

admin.site.register(Category, CategoryAdmin)
Also on page 50 help_text is added. There is a way of overriding this in your admin class but for simple cases you can leave it in the model. I would have thought help_text would have been shifted entirely to forms but I guess there is a case to have some default description in your model to describe the field even if you override it in your form or admin.

On p54 the class Entry is updated to include a prepopulated field slug. Update your admin.py for this.
#admin.py
from django.contrib import admin
from coltrane.models import Category, Entry

class CategoryAdmin(admin.ModelAdmin):
    prepopulated_fields = {"slug":("title",)}

class EntryAdmin(admin.ModelAdmin):
    prepopulated_fields = {"slug":("title",)}

admin.site.register(Category, CategoryAdmin)
admin.site.register(Entry, EntryAdmin)
On p59 the django-tagging app is downloaded.

svn checkout http://django-tagging.googlecode.com/svn/trunk django-tagging

On page 60 the save function for the Entry class uses super which is briefly described. Super was one of those things that seems straightforward to me now, but the first time I saw it, it was like some magicians trick where half the trick is hidden from view, but a rabbit gets plucked impossibly from a hat. For the benefit of those new to python (and programming), I'll show a simple example.
# First we'll have our base class which is usually someone elses code you're going to subclass.
class Container(object):
   """Stuff comes out of containers"""
   def payoff(self, some_object=None):
      if some_object:
         return "dove"
      return

# Now here's the class we might have written
class Hat(Container):
   """Rabbits come out of hats"""
   def payoff(self, some_object=None):
      if some_object == "watch":
         return "rabbit"
      return super(Hat,self).payoff(some_object)

# Here's the example of how we'd use this class

>>> trick = Hat()
# if we put in a watch we get the class we've defined
>>> trick.payoff('watch')
rabbit

# If we put in anything else including nothing at all we get the parent class
>>> trick.payoff('hankerchief')
dove

Pretty straightforward really. I think I originally just got confused by the syntax of passing the name of the class in when you're calling it from the class itself, and the fact that invariably examples I'd seen didn't have the parent class code in front of me.

Onto p61 ignore the class Admin, and then on p69 there is a typo in the regex. The last bit should be (?P<slug>[-\w]+)/$'

On p71 the urls.py is re-written to take advantage of generic views. after d{4} on each entry there should be a closing bracket - ) - and the same again for p73. I've added my urls into my imports in tests.py as well.

Alas no templates are available on the apress website, so you'll have to make your own to take advantage of the generic views.

So finally we come to a bit more python magic with decorators. If you want to read further about python's implementation of decorators and what they can do, have a read of this article, but I figured I might as well include another silly example that is similar in spirit to the way it is used in PDP.

def magician(fn):
    def new_trick():
        print "Cut assistant into bits"
        assist = fn()
        cut = assist.split()[1]
        return [bits for bits in cut]
    return new_trick

@magician
def assistant():
    name = 'lovely assistant'
    print name
    return name

# Get the assistant - you can see the order in which things are called.
>>> a = assistant()
Cut assistant into bits
lovely assistant
# Print what is returned by the magician
>>> print a
['a', 's', 's', 'i', 's', 't', 'a', 'n', 't']
So that's it until part 5. Enjoy.

Wednesday, 27 August 2008

Django testing for absolute beginners

I want to interrupt my series of posts on Practical Django Projects and quickly show how to setup the most basic of tests, and hopefully convince you why taking some small steps and starting with one line of test code can save your sanity especially if you're tracking a code breaking trunk. First the why..

In my previous post I wrote about getting an error in the browser:

ImproperlyConfigured at /admin/
Error while importing URLconf 'cms.urls': 'type' object is not iterable

The problem with this is that the error could be in any one of a number of modules that are imported by urls.py. There are a number of ways code could cause Django to generate this kind of error if you have problems in your code - it just happens that in this case it's newforms admin. Here's the ticket if you're interested.

I spent hours trying to work out this problem because aside from the actual cause of the problem I dug my own grave with a very stupid python mistake. After working out that it was admin files that were causing the problem I renamed one to try and establish which one was causing the problem, but this left the corresponding admin.pyc which python happily kept importing and runserver kept regurgitating the same useless error message. 

Remember, if you remove or rename a python file make sure you remove the corresponding .pyc file.

Ok so you haven't made my stupid mistake, but you're still irritated at having to comment out the admin.site.register function in each admin.py to find your mistake. Well you can save yourself the grief if you create the simplest of tests which in fact would have solved the problem with the redundant .pyc file as well.

Creating a simple tests.py for your project or app will save your sanity and takes almost no time.

Basically every python programmer should know the basic principles of testing using doctest and unittest. Python has the advantage of being able to hack something really quickly, but the temptation is to turn your hack into fully fledged apps without writing tests because it seems quicker. But try coming back to your application a year later and enhancing it. I guarantee you if you have a basic standalone doctest file you'll be up to speed again in no time instead of trying to work out why you'd written some function or trying to debug it. Doctests also seem to help highlight when your app is poorly designed to start with. If it looks wrong in a doctest chances are you want to go back and rework your code. If your app grows in scale then doctests can become unwieldy so you're probably going to want to put in unittests to test more than just basic usage. Ok enough about the principles of why you should write tests, lets look at those basic tests I promised .

First create a tests.py in any project or app directory that holds your apps that get imported by your INSTALLED_ APPS in settings.py. So along side your admin.py, models.py etc. Also if there is no models.py create that as well because the django test runner needs that to know it should load your tests.py.

In your tests.py import whatever modules you have in that directory that have been modified by you. So in the cms example from Practical Django Projects my tests.py would be:
#tests.py
from cms import admin, models, settings, urls, views


That's it! Then go and run python manage.py test  to reap the rewards from your labour. Hopefully it should run a bunch of tests, print OK and end - in which case you're free to go. But lets look at how this one line would have got me out of trouble with the Improperly Configured error if I'd written and run the tests first.

Firstly if I have any syntax error or other basic error in any of my modules, I'll see instantly which one from the test traceback, and all I need to do is fix the error in the module and re-run the tests until I get an OK. Try it. Break your own files and see what happens. Secondly, I would have imported admin in my test so when I renamed the broken admin.py file the test runner would still bring up the error in admin.py alerting me to the fact that it was still being imported. In this case I now want the error to tell me that tests.py can't import the admin module. Once it does this I then know I just need to update my tests.py removing the admin import.

Ok that's all well and good if I was going to write a unittest but how about doctests. Well here's the most basic doctest that does the same thing.

#tests.py
tests = r"""
>>> from cms import admin, models, settings, views, urls

"""

__test__ = {"cms_tests":tests}
Ok so thats enough from me. Go have a look at the official django docs on testing and this post and start writing some proper tests.

Monday, 25 August 2008

3. Customizing the Simple CMS

This is part 3 of a series of posts on James Bennett's excellent Practical Django Projects. The table of contents and explanation can be found here.

Improving the Search Function with Keywords

On page 34 when defining your first model class, remove the Admin class to replace it with new a separate newforms admin file (hereafter just admin). If I've ignored code that works just fine I just use ... to show this.
class SearchKeyword(models.Model):
...
   #REMOVE THIS
   class Admin:
      pass
Instead of the basic class Admin you create an admin.py file in the search folder. The simplest possible admin configuration is:
from django.contrib import admin
from cms.search.models import SearchKeyword

admin.site.register(SearchKeyword)
Registering the SearchKeyword allows it to be autodiscovered and included on the admin site. On p36 the SearchKeyword model admin is improved. Firstly remove core = True, the admin now uses a checkbox instead so this is not used any more. Then create an admin.py in the cms folder.
from django.contrib import admin

from cms.search.models import SearchKeyword
from django.contrib.flatpages.models import FlatPage
from django.contrib.flatpages.admin import FlatPageAdmin

class SearchKeywordInline(admin.StackedInline):
    model = SearchKeyword
    max_num = 3
    extra = 1

class MyFlatPageAdmin(FlatPageAdmin):
    inlines = [
    SearchKeywordInline,
    ]

admin.site.unregister(FlatPage)
admin.site.register(FlatPage, MyFlatPageAdmin)

The custom MyFlatPageAdmin inherits and adds the inline SearchKeyword form. To register your custom FlatPage admin you need to unregister the original first. Finally you can delete the search.admin.py file if you only want the inline form.

Caution - Improperly Configured Admin

While writing this post I got myself in a knot when I got this error:

ImproperlyConfigured at /admin/
Error while importing URLconf 'cms.urls': 'type' object is not iterable

Clearly this unhelpful error message is causing all sorts of grief as people migrate code to the new admin classes, but I'm sure relief won't be too far off. For the purposes of learning it's actually not to difficult to resolve once you know what you're meant to be looking for.

The first step to resolving the Improperly Configured error would be to comment out your admin.autodiscover() in urls.py. If the error goes away (though admin won't function correctly) then it's an admin class causing the problem. Uncomment it again then basically find the misbehaving admin class by commenting out the admin.site.register(FooAdminClass) functions in each admin.py until you find the one that is misbehaving.

As it was I had tried to register the SearchKeywordInline StackedInline admin class in it's own right instead of just leaving it as part of the custom FlatPagesAdmin class.

All done with the basics of the new admin? Have a read of this blog post and screencast from oebfare on migrating to the newforms admin.

2. Your First Django Site: A Simple CMS

This is part 2 of a series of posts on James Bennett's excellent Practical Django Projects. The table of contents and explanation can be found here.

Configuring Your First Django Project

Choosing an editor for python is a personal choice. As a beginner you can do worse than just use IDLE which comes with most python distributions (separately on os x)  or pythonwin if you are on the windows platform. These have the advantage of autocompletion of function arguments, but once you're comfortable you're likely to move onto something that manages a project nicely and is more developed. I use Textmate on OS X but there's a full list here. If I have been working on nonpython projects I miss not having autocomplete in Textmate but sometimes I'll just keep IDLE open just for that.

Putting Together the CMS

On p13 is where the first deviation from the older django kicks in. In your settings.py add 'django.contrib.flatpages.middleware.FlatpageFallbackMiddleware', (don't forget the comma) to your MIDDLEWARE_CLASSES. Django 1.x includes a new forms admin which separates out the administrative configuration from the models. By default the new urls.py includes:
 # Uncomment the next two lines to enable the admin:
 # from django.contrib import admin
 # admin.autodiscover()  
 # Uncomment the next line for to enable the admin:
 #   (r'^admin/(.*)', admin.site.root),
Which you will want to uncomment to enable the admin
Instead of
 
# (r'^admin/', include('django.contrib.admin.urls')),
On page 15 you must make sure you edit the example.com site instead of adding a new site. You might have noticed in your settings.py the SITE_ID = 1. If you add a new site 127.0.0.1:8000 then that will have a SITE_ID of 2, and in the following section flatpage views filter by default on the current site which is 1.

The urls.py is where beginners face one of their biggest hurdles. On the face of it regular expressions are difficult to craft without practice, and implementations subtly vary from language to language. Time to add to the bookshelf with another bible, Mastering Regular Expressions which is probably more than you'll ever need. On the free front A.M. Kuchling has a great python how-to. One aspect of regexes that irritates is trying to find tools that you can use to test them easily while you are building your own. A few of the tools I have found for OS X have been inconsistent only modestly useful. One online recommendation is this site though I haven't used it yet.

A Quick Introduction to the Django Template System

Once you've finished reading this you might want to head over to Jeff Croft and read his post on default templates.

1. Welcome to Django

This is part 1 of a series of posts on James Bennett's excellent Practical Django Projects. The main post can be found here. This is not a critique of James's book, but rather a collection of notes and code revisions to accompany it, both updating it for Django 1.x and things beginners may want to consider. The headings follow the subheadings in the book.

Say Hello to Python

If you haven't already bought a book on python and want some other recommendations to the ones given then I highly recommend Learning Python 3rd edition by Mark Lutz. Both the 1st and 2nd very thumbed editions took a long time for me to grow out of. The only thing I wish had been in Learning Python was a few modern software engineering principles such as test driven development. In fact I almost wish the first program was a doctest
>>> print 'Hello World'
Hello World
It took me a while to realise that learning to program without testing is like learning to drive without learning how to use the brake.

Looking Ahead

For learning Django, sqlite is the easiest entry point, but if you do decide to move your applications into the real world chances are you're going to want to install either Mysql or Postgresql. Just don't try to make a decision based on transaction speed, it's a futile exercise. Personally I think Mysql is easier to use and quicker to learn sql on than Postgresql if you're accessing your data outside Django, but once you're more experienced I think the balance tips to Postgresql as the more mature database, but for specific requirements each one has it's own advantages and disadvantages. I have happily used both in production but here's my take on the broader differences between them:

Mysql
 + The documentation has always been excellent and I think better than Postgresql
 + In some ways Mysql feels like Django in that they have made pragmatic choices to make things easier for rapid development. The prime example is the REPLACE statement which has no equivalent in Postgresql and statements like INSERT IGNORE.
 + Replication is easy to setup.
 + Works better with external odbc destinations such as Excel and Crystal Reports (the latest Crystal includes official support for Mysql).
 + The freely available tools are easier to use than those that are available for Postgres
 + A wide range of pluggable storage engines for different purposes

Postgresql
 + Stored procedures are much more mature than Mysql and you can use Python
 + If you ever have to slice and dice your data for reports with views then Postgresql is by far the best choice. Try writing a view of a view and you will quickly fall into a blackhole with Mysql.
 + For large data sets you can have partial indexes to speed up queries on massive tables.
 + In general Postgresql seems less forgiving when you try to do things you shouldn't with SQL, which is painful when learning but less likely to lockup your server.

For further reference a good up to date comparision is on wikivs

Practical Django Projects

June 2009 News Update: The second edition of this book has been released!

The 2nd edition of Practical Django Projects has been released making this series of posts redundant!

Notes and a practical companion

I've been reading James Bennett's excellent Practical Django Projects (PDP), which uncritically I think is the best python based book I've read since I picked up Mark Lutz's Learning Python in 1999. Unfortunately for James the arrival of Django 1.x breaks code examples in the book, which may frustrate the target audience for the book who are likely to be new to Django, and possibly python. On the other hand, if the reader has some basic prior python experience, then there is nothing like learning an application when you have to update the code, and the users mailing list, and Django docs are excellent resources to this end.

But for those budding Django users who might be less inclined to wade through google searches on the django-users list, and to avoid others repeating themselves I thought it would be worth posting some revisions of the code samples as I work through the chapters, and also add some footnotes to James's text where I think they'd be useful to other readers. As I do each chapter, I'll add in a new post if I have anything to add and create a link from here.
  1. Welcome to Django 
  2. Your First Django Site: A Simple CMS
  3. Customizing the Simple CMS
  4. A Django-Powered Weblog
  5. Expanding the Weblog
  6. Templates for the Weblog
  7. Finishing the Weblog
  8. A Social Code-Sharing Site
  9. Form Processing in the Code-Sharing Application
  10. Finishing the Code-Sharing Application
  11. Writing Reusable Django Applications
  12. (I will not be doing a post for this chapter)
Now for the disclaimer. As someone who has had no prior experience with Django, I don't claim any authority on the 'Django' way, and invite anyone to comment if they have a better solution.

UPDATES
29-Aug-08 - Rev Pt4. Django-tagging has been updated and the trunk works with 1.0 Beta 2
5-Sep-08 - Rev Pt5. Updated to modify category view

Hedged Down