Django 1.3 & South

Running the latest Django Release Candidate, I noticed that all of my South data migrations were failing.

Debugging via pdb led me to a a recent change in which the database was being flushed after south ran, but before the tests ran. I found a changeset that was committed after Django 1.3 beta-1, #14661 that introduced the flush to correct an issue with MySQL. It is a documented as a backwards incompatible change, which will prevent SQL fixtures (and unfortunately South data migrations).

After talking with Andrew Godwin at the PyCon Sprints, we decided to override the flush operation during testings. It is an unfortunate hack, but allows data migrations on Django 1.3.

django-durationfield v0.3.3

I just released an implementation of DurationField for Django to PyPi.

The latest release uses datetime.timedelta for its internal representation of durations (thanks to Paul Oswald) and has support for South thanks to Wes Winham.

The documentation is now hosted on ReadTheDocs and the package data is available on Django Packages.

After a discussion with a few Django core developers, it seems like keeping the DurationField implementation as a separate, reusable application is the preferred option. By staying independent, we keep the ability to make changes independent of the Django release cycle. Additionally, we avoid adding bloat to Django-core. Since not all databases implement an interval or duration data type (PostgreSQL does), django-durationfield is in some ways a hack by using a bigint datatype to store integer instances of timedelta.

jsonpickle 0.3.1 Released

jsonpickle, the powerful library for serializing complex object graphs in Python to JSON, had a major milestone this week with the official release of 0.3.1, available on PyPi with documentation and full release notes at We have migrated from the Google Code site to the new Github site at

This release represents nearly a year of development from multiple contributors. Some of the major highlights of the release include supporting a wider variety of objects, supporting the pickle protocol's set and get state methods, and allowing the use and addition of any Python JSON backend (e.g. demjson, simplejson, django.util.simplejson, etc.). Please be aware that backwards compatibility for the 0.2.0 format JSON is not guaranteed.

In the past year jsonpickle has done well, with nearly 2500 downloads of the 0.2.0 release from Google Code, not including the bundled distribution of jsonpickle in tools such as FireLogger and git-cola. jsonpickle is currently available in the Gentoo repository and working its way through the Fedora and Debian repository processes.

I thank our contributors, including David Aguilar, Dan Buch, and Ian Schenck for their massive improvements to jsonpickle. I also thank everyone who has submitted bug reports and shared thoughts on our mailing list. Finally, I thank the distribution managers who have worked to package jsonpickle for their various distributions.

Please try the new version, submit bug reports, and even fork the project on Github.

HTML to reStructuredText in Python using Pandoc

During the conversion of my blog from Wordpress to a custom Django-based system, I wanted to move from HTML markup to reStructuredText (partly to make it easier to publish Sphinx documentation to my blog).

While it is dead simple to convert reStructuredText to HTML, going the other way is more difficult. Luckily, Pandoc, the swiss army knife for converting between markup formats, can do a nice job converting HTML to reStructuredText.

I wrote a custom Django Command to parse a Wordpress XML export file and store the blog entries. The relevant code to convert HTML to reStructuredText is very simple. It simply makes a subprocess call to the Pandoc command and retrieves the command's output. Make sure you have Pandoc installed (in Ubuntu, sudo apt-get install pandoc will work).

import subprocess
def html2rst(html):
    p = subprocess.Popen(['pandoc', '--from=html', '--to=rst'],
                         stdin=subprocess.PIPE, stdout=subprocess.PIPE)
    return p.communicate(html)[0]

reStructuredText Widget in Django Admin

While Django has great support for rendering standard markup languages, sometimes it can be difficult editing documents using a markup in the Django Admin. Several others (1, 2, 3) show how easy it is to edit Markdown, Textile, and even HTML, in the Django Admin using a WYSIWYG editor, such as markItUp! or TinyMCE.

Unfortunately, reStructuredText is not well support by most WYSIWYG editors. Nevertheless, we can improve the experience of editing reStructuredText in the Django Admin. One of the biggest improvements is switching the Textarea to use a monospace font to avoid issues caused by the heading underlines being too short. We can also customize the size of the Textarea.

In our app/ file, we can add a ModelForm which overrides the field that has the reStructuredText content (in this case, description). We then use this ModelForm as the form in our subclass of ModelAdmin. Finally, we indicate that the ModelAdmin subclass is associated with the specific model that contains the reStructuredText content.

For the reStructuredText content field, description, we change the size to a width of 80 characters and the font to a monospace family. Additionally, we add a quick link to the reStructuredText Quick Reference.

from django import forms
from django.contrib import admin
from app.models import Entry

class EntryAdminForm(forms.ModelForm):
    description = forms.CharField(widget=forms.Textarea(attrs={'rows':30,
                                  help_text='<a href="">reStructuredText Quick Reference</a>')
    class Meta:
        model = Entry

class EntryAdmin(admin.ModelAdmin):
    form = EntryAdminForm, EntryAdmin)