Django Menuing System

On most of the websites that I've built with Django, I have had a desire to be able to manage little elements of the website from the Django administration screen without having to touch my templates. My intent is for the templates to become the presentation vehicle, with anything that matters being built out of the Django databases.

One such thing that I want to keep out of my templates is navigation. Sure, the template has a place for navigation (including an empty

    ), but the contents of my navigation bars are driven by a dynamic Django application.

    The application has but two files: the models and a template tag.

    First up is the model file, menu/models.py:

    from django.db import models
    
    class Menu(models.Model):
        name = models.CharField(maxlength=100)
        slug = models.SlugField()
        base_url = models.CharField(maxlength=100, blank=True, null=True)
        description = models.TextField(blank=True, null=True)
    
        class Admin:
            pass
    
        def __unicode__(self):
            return "%s" % self.name
    
        def save(self):
            """
            Re-order all items at from 10 upwards, at intervals of 10.
            This makes it easy to insert new items in the middle of 
            existing items without having to manually shuffle 
            them all around.
            """
            super(Menu, self).save()
    
            current = 10
            for item in MenuItem.objects.filter(menu=self).order_by('order'):
                item.order = current
                item.save()
                current += 10
    
    class MenuItem(models.Model):
        menu = models.ForeignKey(Menu)
        order = models.IntegerField()
        link_url = models.CharField(maxlength=100, help_text='URL or URI to the content, eg /about/ or http://foo.com/')
        title = models.CharField(maxlength=100)
        login_required = models.BooleanField(blank=True, null=True)
    
        class Admin:
            pass
    
        def __unicode__(self):
            return "%s %s. %s" % (self.menu.slug, self.order, self.title)
    

    Next is a template tag that builds named or path-based menus - menu/templatetags/menubuilder.py:

    from menu.models import Menu, MenuItem
    from django import template
    
    register = template.Library()
    
    def build_menu(parser, token):
        """
        {% menu menu_name %}
        """
        try:
            tag_name, menu_name = token.split_contents()
        except:
            raise template.TemplateSyntaxError, "%r tag requires exactly one argument" % token.contents.split()[0]
        return MenuObject(menu_name)
    
    class MenuObject(template.Node):
        def __init__(self, menu_name):
            self.menu_name = menu_name
    
        def render(self, context):
            current_path = template.resolve_variable('request.path', context)
            user = template.resolve_variable('request.user', context)
            context['menuitems'] = get_items(self.menu_name, current_path, user)
            return ''
    
    def build_sub_menu(parser, token):
        """
        {% submenu %}
        """
        return SubMenuObject()
    
    class SubMenuObject(template.Node):
        def __init__(self):
            pass
    
        def render(self, context):
            current_path = template.resolve_variable('request.path', context)
            user = template.resolve_variable('request.user', context)
            menu = False
            for m in Menu.objects.filter(base_url__isnull=False):
                if m.base_url and current_path.startswith(m.base_url):
                    menu = m
    
            if menu:
                context['submenu_items'] = get_items(menu.slug, current_path, user)
                context['submenu'] = menu
            else:
                context['submenu_items'] = context['submenu'] = None
            return ''
    
    def get_items(menu, current_path, user):
        menuitems = []
        for i in MenuItem.objects.filter(menu__slug=menu).order_by('order'):
            current = ( i.link_url != '/' and current_path.startswith(i.link_url)) or ( i.link_url == '/' and current_path == '/' )
            if not i.login_required or ( i.login_required and user.is_authenticated() ):
                menuitems.append({'url': i.link_url, 'title': i.title, 'current': current,})
        return menuitems
    
    register.tag('menu', build_menu)
    register.tag('submenu', build_sub_menu)
    

    Using this menu system is relatively easy:

    1. Filesystem setup

      1. Create a directory called menu in your Python path
      2. Drop models.py (above) into the menu folder
      3. Create a directory called templatetags inside the menu folder
      4. Copy the menuubilder.py (above) into the templatetags folder
      5. Create a blank file called _init.py_ and put a copy in each of your menu and templatetags folders
    2. Django setup

      1. Add menu to the INSTALLED_APPS list in your settings.py
      2. Run ./manage.py syncdb to create the relevant database tables
    3. Data setup (using Django's Admin tools)

      1. Add a Menu object for each menu set you wish to use. Give static menus (eg, those that are the same on each page) a slug such as main, footer or sidebar. For dynamic menus, that display different contents on different pages, add a base URL. A menu with a base URL or /about/ might contain links to your philosophy, your team photos, your history, and a few policies - but only when the user is visiting a page within /about/.
    4. Template setup

      1. In your template, wherever you want to use a particular named menu, add this code:

        {% load menubuilder %}{% menu main %} {% for item in menuitems %}
      • {{ item.title }}
      • {% endfor %}

      1. Replace main with the name of a static menu (eg footer or sidebar from the above example)
      2. For dynamic URL-based menus, add this code:

      {% load menubuilder %}{% submenu %}{% if submenu %}

      {% endif %}

      1. If a menu has been set up with a BASE URI that the user is currently seeing, it will be displayed. If no such menu exists, nothing will be displayed.

    I use named menus for link bars in header and footers. I use URI-based menus to display a sub-section navigation, for example within the about area on a website or within the products section. A URI-based menu will only be displayed with the user is within that URI.

    So there you have it. Database driven menus that let you build easy static or URI-based menus and submenus. If you look carefully there is code to only display links to logged in users if required, and the page that is currently being viewed is tagged with the current CSS class so it can be easily styled.

    If all of that is confusing, stay tuned for my Basic Django-powered website in a box that will pull together a number of elements such as this into a simple to install package you can use to get a website up and running quicksmart - with the fantasmical automated Admin screens to control it all.

Wanted: Engaged Djangonauts

I've spent the last year or so building a Django-powered bridal gift registry service. It allows you to easily list desired gifts in a simple online format, for your guests to select and purchase for you.

It works pretty well - my wife let me use it for our own wedding, afterall. It's also stood up to a half-dozen other weddings, who have all provided fantastic feedback which I've put into place.

I'm just about ready for launch, but I'd like to do a final round of live testing. So, I'm opening it up to Django users worldwide.

The criteria?

  • Be getting married in the short-term future
  • Be prepared to provide feedback
  • Be prepared to use public beta software (However, I believe it's stable enough!)
  • Have a [wife|husband]-to-be who is happy to be involved :)

In return I'll provide a premium account (value: $40AUD) for you to use from day one.

Interested? Just e-mail me at ross at this domain and I'll provide a signup URL. You're welcome to sign up and check it out before committing to using it for your wedding, of course - however if you choose not to use it, I'd love to know what WAF (that'd be wife-acceptance-factors) come into play in making your decision, so I can improve it as much as possible!

DjangoPoweredSites Grows Up

After some e-mail discussions with Jacob Kaplan-Moss from the Django team, I've moved all of the sites listed on the old DjangoPoweredSites wiki page to Djangosites.org.

It took a fair bit of time to weed out some not-working pages and expired domains (which have, of course, been purchased by spammers and link farmers), and a small investment in extra WebThumb credits to allow me to take screenshots of a fresh 500+ websites, but I got there in the end.

We'll now be closing the DjangoPoweredSites wiki page for good, as it has become redundant and is not the best way to peruse 800+ websites.

What we're left with is a fantastic directory of websites powered by Django. I'll be putting together a 'Featured' list shortly, but in the meantime I need a little bit of help from the community.

All sites scraped from the wiki page are currently unowned. I encourage you to check out the list of unclaimed websites, and e-mail me to let me know which sites are yours so you can receive due credit. There are some great websites listed there, so it'd be fantastic to be able to show off who built them.

Thanks to Jacob and the rest of the Django team for getting behind this initiative.

Easy Multi-Part E-Mails with Django

Every time I send e-mail with Django I seem to do it slightly differently, especially when it comes to sending HTML-based emails.

So, I did what any good Djangonaut does and wrote a quick function that wraps up sending of a multi-part HTML / Text e-mail from templates you create.

Firstly, create your e-mail templates. You'll need both a HTML and a plain-text template; for the HTML template I strongly recommend the use of Premailer, which puts all CSS definitions inline.

Here's a sample HTML template - as it would come out of Premailer. We'll pretend it's in the emails template directory, with a name of newsite.html.

<h1 style='font: Arial; font-weight: bold; font-size: 15pt; color: #006;'>Thanks!</h1>

<p style='font: Arial,Helvetica; size: 10pt;'>Hello, {{ username }}.</p>

<p style='font: Arial,Helvetica; size: 10pt;'>Thank you for submitting your website, titled <strong>{{ title }}</strong>, to our listing.</p>

<p style='font: Arial,Helvetica; size: 10pt;'>We'll verify it, take a screenshot, and publish it within the next few days.</p>

<p style='font: Arial,Helvetica; size: 10pt;'>Regards,</p>

<p style='font: Arial,Helvetica; size: 10pt;'><em>Your Favourite Webmaster.</em></p>

And here's that same e-mail, in plain-text format - also in the e-mails folder, called newsite.txt.

Hello, {{ username }}.

Thank you for submitting your website, titled '{{ title }}', to our listing.

We'll verify it, take a screenshot, and publish it within the next few days.

Regards,

Your Favourite Webmaster.

Those who are familiar with Django's template language will see that this template uses two variables, username and title. Let's populate them, shall we?

template_context = {
    'username': 'joecitizen',
    'title': 'My Movie Review Site',
    'url': 'http://www.moviewreviews.dom',
}

So far we have an e-mail message template in both HTML and text formats; all we need now to send an e-mail is a subject and a recipient.

recipient = 'joe@example.dom' # ['joe@example.dom',] would also be suitable
subject = 'Site submission: {{ url }}'

That's pretty straightforward - our subject also contains Django template variables, in this case the URL from our template context we created earlier.

Now it's time to send the message. I have been using this function with much success:

def send_multipart_mail(template_name, email_context, subject, recipients, sender=None, fail_silently=False):
    """
    This function will send a multi-part e-mail with both HTML and
    Text parts.

    template_name must NOT contain an extension. Both HTML (.html) and TEXT
        (.txt) versions must exist, eg 'emails/public_submit' will use both
        public_submit.html and public_submit.txt.

    email_context should be a plain python dictionary. It is applied against
        both the email messages (templates) & the subject.

    subject can be plain text or a Django template string, eg:
        New Job: {{ job.id }} {{ job.title }}

    recipients can be either a string, eg 'a@b.com' or a list, eg:
        ['a@b.com', 'c@d.com']. Type conversion is done if needed.

    sender can be an e-mail, 'Name <email>' or None. If unspecified, the
        DEFAULT_FROM_EMAIL will be used.

    """
    from django.core.mail import EmailMultiAlternatives
    from django.template import loader, Context
    from django.conf import settings

    if not sender:
        sender = settings.DEFAULT_FROM_EMAIL

    context = Context(email_context)

    text_part = loader.get_template('%s.txt' % template_name).render(context)
    html_part = loader.get_template('%s.html' % template_name).render(context)
    subject_part = loader.get_template_from_string(subject).render(context)

    if type(recipients) != list:
        recipients = [recipients,]

    msg = EmailMultiAlternatives(subject_part, text_part, sender, recipients)
    msg.attach_alternative(html_part, "text/html")
    return msg.send(fail_silently)

Lastly, to put together the previous example:

from multipart_email import send_multipart_email # Assuming, of course, you saved the above functoin in multipart_email.py

send_multipart_mail('emails/newsite', template_context, subject, recipient)

What this does, is uses our existing template context, subject, and recipient, to send a HTML and plain-text e-mail based off the templates we built earlier. Note that it automatically appends .html or .txt as required. Hopefully the extra documentation in the docstring is useful.

Recipients who can read HTML e-mail will see your nicely formatted HTML e-mail message, while everyone else will see the plain-text alternative.

Slowly, this method is finding it's way into each of my Django applications. Please, just don't overdo the HTML messages!

Django: Multiple Aliases for a Single Website

In these days of cheap domains, it's often desirable to own multiple domains for a single website. You've probably got each of the .com, .net and .org domain names, along with a country-specific domain. You want each of these to present exactly the same website to the world, but good design says that each web page should have one, and exactly one, URL. So what's the best way to serve this up without having an Apache config for each domain?

I've come across this whilst building a website recently whereby the primary domain is mydomain.com.au, while I've got secondary domains in other popular TLD's to try and reduce domain squatting and the like.

One option is to configure an Apache virtual host for each domain, which serves up a static redirect. Another is to have Apache aliases for the main host, so each of the domains serves up the same content. This works, but leaves each page with multiple URL's.

My solution is to set up Apache aliases, and use a Django middleware to identify any requests that aren't for the main domain name, redirecting them as they're found. The middleware code I use is as follows:

from django.http import HttpResponsePermanentRedirect

class ValidateHostMiddleware(object):
    """
    Redirect all requests for a domain other than mysite.com.au
    """
    def process_request(self, request):
        if not request.META['HTTP_HOST'].endswith('mysite.com.au'):
            return HttpResponsePermanentRedirect('http://www.mysite.com.au%s' % request.path)

This is nice and simple, and a useful way of having multiple domains (possibly increasing your virtual 'geographical spread') but keeping your search-engine optimisation efforts intact.

Update: Thanks to a note from Brice Carpenter, I've updated the code to do a permanent HTTP redirect (code 301) rather than a temporary (302) redirect. I've also added code from my live environment that sends the visitor to their request path on the new domain - so hitting www.mysite.com/about/policies/ refers the user to www.mysite.com.au/about/policies/.

Django: A Diverse Community

Scott Barnham, one of the guys behind the recently-launched Django Gigs website, has posted some statistics from visitors to the Gigs website over the past few days. I've put together some similar stats for the Django Sites website, which has been online for a few months now.

Note: All figures are from Google Analytics, and percentages are rounded to make life easier.

Summary

Operating Systems

Windows 60%

Linux 21%

Macintosh 18%

Others 1%

The operating system split is very much skewed towards Windows, almost all of the Windows users on Windows XP. It's nice to see higher-than-average (compared to my other sites) usage of Linux & Macintosh.

Browsers

Firefox 66%

Internet Explorer 16%

Safari 7%

Opera 5%

Others 6%

No surprises here. Firefox figures are very similar to the Gigs site, with Safari & IE being a bit different. I'm guessing the higher usage of IE shows the target audience of DjangoSites - people who are looking for what Django is capable of, perhaps newer-comers to the Django world rather than long-term stable mates who may be more likely to use Firefox or Safari on their shiny Macbook Pro.

Geographical Areas

North America 30%

Western Europe 18%

Eastern Europe 9%

Southern Europe 9%

Northern Europe 9%

South America 6%

Eastern Asia 4%

Australia & New Zealand 3%

Elsewhere 12%

Again, this isn't really too surprising. There is a very long-tail (and I'm not overly sure Google Analytics needs to break down Europe & Asia into so many sub-continents), easily visible with "Elsewhere" covering 12% of the visits.

Stuff I Found Interesting

  • Bolivian visitors viewed 20 pages/visit, followed by 18/visit for Oman. Everyone else was miles behind this.
  • 52% of traffic is from referrals, most of it from the Django website, an article in Smashing Magazine, and Wikipedia.
  • 30% of traffic is search-engine generated, typically from searches for 'django sites' - and somehow, including a few searches for 'Auction'.

Hopefully these numbers are interesting to people looking to target their offerings to the Django audience. Do they match your sites?

Mixing OpenID into Django's authentication system

NOTE: This code is now outdated, and it's certainly not the best way to do OpenID in Django. I recommend you take a look at django-authopenid, a fantastic registration system that combines Django's authentication framework with OpenID sign-in. Ross, 17th April 2008

According to the OpenID website, from a consumers point of view OpenID is "the elimination of multiple user names and passwords and a smoother, more secure, online experience. ". What it provides is a single identity for you to use at multiple websites. Instead of having a username and password for each website you peruse, you have an identity (usually a URL to your blog or an OpenID provider) that you use to login. The only password you have to remember is that of your OpenID provider - and you don't have to provide your password to any websites you visit.

Simon Willison has been a fantastic campaigner for OpenID, especially for integration with Django. He's written the fantastic django-openidconsumer package which provides the framework for a Django Application to act as an OpenID consumer (that is, people login to your Django app using their OpenID).

Simon's package creates a new OpenID object within your application, but is unrelated to the existing authentication system. So what I've done is mixed some glue to pull together these fantastic standalone applications:

The 'glue' provides a useful process flow for new users to your website. It lets people register with a username/password as they would have before OpenID, lets people login with OpenID's, and keeps it all intertwined. The basic functions are as follows:

  • A new user can sign up with a username/password, using django-registration
  • A new user can enter an OpenID, which is authenticated before creating a standard Django user. The account is verified using django-registration's email checker.
  • A user who is already logged in with a username/password, can login again using an OpenID, which is automatically associated with their username
  • A user can login with either their username/password (if they have one), or with their OpenID(s) (if they have any), and you'll always see them as 'request.user' - the same as if they had signed up with a username/password
  • This gives users a choice when they sign up and lets them change their mind by adding or removing OpenID's from their account as they wish.
  • Why do e-mail verification when users sign up with an OpenID? Because OpenID doesn't guarantee a users identity. It's merely a replacement for a username & password. E-Mail verification helps cut down on automated registrations, and ensures users are providing you with a valid contact address.

How would I use this in the real world? For sign-ups, I would show one form, clearly divided, asking a user to enter their OpenID if they have one, or otherwise enter their username/password/e-mail address. For logins, show users a username/password box with a link to toggle an OpenID entry field. If the user logs in with an OpenID, set a cookie so you can remember them in the future, showing them the OpenID login by default rather than the Username-driven login. Make it as easy as possible for people to adopt this fantastic technology.

I'm not using this code in a live environment yet, as it needs more testing. However, I've put a testbed online and made the source code available.

I would love it if you could try out this code both on my server and on yours, and provide feedback on the flow from a users point of view. I plan on cleaning up the code significantly before packaging it into something that is safe to use in a production environment.

The online demo is at http://openid.rossp.org. There is a link to the source code there.

Note: The database isn't accessible via the web, and it's a database set up specifically for this demonstration. I will NOT be extracting lists of usernames, passwords, e-mail addresses, openid's, etc. I may pu tup some usage statistics later, but certainly nothing identifiable. Please feel safe using this.

Validating a Username via jQuery with Ajax

It all starts when John hits your website and clicks the big 'Register' link. John types his name, '_John_' into the username box, and hands over his e-mail address and password (unless you're cool and hip, and you let him sign up using his OpenID) and hit 'Submit', just like every other website he's signed up to in the past.

Except this time, somebody else called John (what are the chances, eh?) has already signed up using that username, so after waiting a few seconds John sees an error message asking him to select another username. He types a new username and tries 'Submit' again, unsure as to whether his new selection will be suitable. So we fix this problem easily - we tell your users, while they're entering their username, whether their selection is available.

To achieve this we're going to use jQuery with it's fantastic Ajax support.

To get started, we create a simple view in Django that confirms the presence of a given username. This view will be accessible at /check_username/; and will expect a username via a GET paramater.

def checkusername(request):
    from django.contrib.auth.models import User
    from django.http import HttpResponse
    username = request.POST.get('username', False)
    if username:
        u = User.objects.filter(username=username).count()
        if u != 0:
            res = "Already In Use"
        else:
            res = "OK"
    else:
        res = ""

    return HttpResponse('%s' % res)

You'll now find that accessing /checkusername/?username=john_ will return 'Already in Use' or 'OK', as required.

The next thing to do is access it from within your registration form. I use James Bennett's fantastic django-registration with a customised template. This means I don't have to alter the registration code at all! I've added a little element just next to the username field, which we'll update shortly. The username field on my form looks like this:

<dt><label for="id_username">Username:</label></dt>
<dd>{{ form.username }} <span id='username_status'></span> {% if form.username.errors %}<span class="formerror">{{ form.username.errors|join:", " }}</span>{% endif %}</dd>

We've got an empty there called usernamestatus_. Now, when the user types a username, we want to check that username via background AJAX call and update the <span%gt; appropriately.

In the header of your registration form (I use a block in my templates called {% block extrahead %} which is inside the of my base.html) you will need to add a handful of lines of JavaScript:

<script type='text/javascript' src='/media/jquery.js'></script>
<script type='text/javascript'>
var previous_username = '';
var in_ajax = 0;
function checkUsername() {
    username = $("#id_username").val();
    if ((previous_username != username) && (username != '') && (in_ajax != 1)) {
        in_ajax = 1;
        $("#username_status").html("<img src='/media/busy.gif' />");
        $("#username_status").load('/check_username/', {username: username}, function() {in_ajax = 0;});
    }
    previous_username = username;
}
$(function() {
    setInterval("checkUsername()", 1000);
});
</script>

This code is relatively simple. Firstly, it loads jquery.js, which I hope by now you've downloaded and placed into a suitable media directory. Then, using the last 3 lines, it causes the checkUsername() function to be called every second. This function does an AJAX request to our checkusername_ view and puts the response into our usernamestatus_ .

We do a few niceties here, too. Firstly, we make sure the username isn't the same as when we last checked it (no point in checking the same username every second - it would increase server load with no real benefit). We also put a 'busy' image into the status area whilst doing the AJAX call - I use a simple image from ajaxload.info.

Net result? The user gets real-time feedback on whether their username is available, so they don't have to think as hard when the page re-loads with an error message.

Note that this won't stop the user submitting a form with an invalid username - if they do submit it, they'll get an error message as per usual. All this function does is provide some nice feedback to users who wish to use it. Signing up to a website should be easy, after all.

DjangoSites: We Want YOU!

I've just approved another batch of sites bringing the total to 260. I've also had to delete a few sites that were submitted due to a number of reasons: A handful were submitted that are obviously not Django (providing your link as /index.php is a give away) and a few have been submitted that are inaccessible URL's.

If you've got a website built with Django, we'd love for you to submit it. There are about 600-700 sites listed on the wiki page, if we can get most of those moved over to DjangoSites.org it'd be fantastic. It's a great way to show off what's out there in the wild that's powered by Django, and a great way to get feedback on your sites.

For those learning Django, it's also a great way to see how others have built their sites. Our listing of sites with publicly available source is growing every week, and is a great way to view how different problems have been tackled.

In the meantime, maddiin and I are working on a repository for Django-powered applications. There are plenty of projects out there (especially on Google Code), however having listed together in one place can only be a good thing. At a glance, you'll be able to see what version of Django and Python you need for a given application, and you can set your user preferences to only show applications that run on your version of Django so you aren't constantly looking at potentially incompatible applications. Keep your eyes here for more upcoming information about this.

Djangosites Updates

Last week I launched djangosites.org, a new website to show off other websites built with Django. Uptake has been great (as I write this there are 125 websites listed) and feedback has been even better.

Firstly there are more RSS feeds. You can view RSS feeds for latest entries, particular tags (eg business), or individual authors. The 'RSS' button at the top-right of most pages links to an RSS feed of your current view, more will be coming soon.

Next, listings with source-code available are now more easily identifiable from the listings by a new icon: (a good example of this is the blog tag listing which has a few listings with source-code available)

We've also done a fair few minor cosmetic changes, eg signup emails now come from a sane email address, and the signup/login forms match the rest of the site

Any other suggestions are more than welcome, in the meantime maddiin and I are working on our next Django community project! Stay tuned for more details.

Introducing Djangosites.org

Until now, if you wanted to see a few sites that were built with Django you had to wade through a list of a few hundred links on a wiki page. The links were somewhat organised into categories, but no matter who you ask that's a boring job.

Djangosites.org has been built as a showcase of what's out there in Django Land.

We hope it lets you easily see what's capable with the Django framework. Feel free to add your sites, post comments on other sites, and vote for your favorites. What I'd really love is if some of the Django heavyweights could post some sites up - it would kick ass if this can eventually replace the DjangoPoweredsites wiki page.

Thanks to maddiin for the site design, and to the django-users mailing list for feedback and suggestions. If you have any feedback please don't hesitate to email it to me at ross at this domain.

Using The WebThumb API with Python

First things first: Get an API key from bluga.net by signing up for an account.

Next, drop the following code into webthumb.py and enter your API key.

"""
Python interface to Webthumb API (see http://bluga.net/webthumb/)

By Ross Poulton - www.rossp.org

License: Use this how you like, just don't claim it as your own because
         that isn't cool. I'm not responsible for what this script does.

Usage: Define WEBTHUMB_APIKEY with your API key, as per the above URL.

Then, just call get_thumbnail(url, output_path). It will return true on
success, false on anything else.

An optional third parameter can be passed for the image size.
"""

import time
import os
import httplib

import xml.dom.minidom
from xml.dom.minidom import Node

WEBTHUMB_APIKEY='Enter your webthumb API key here'

WEBTHUMB_HOST='webthumb.bluga.net'
WEBTHUMB_URI='/api.php'

VALID_SIZES = (
    'small',
    'medium',
    'medium2',
    'large',
)

def get_thumbnail(url, output_path, size='medium2'):
    if size not in VALID_SIZES:
        return False

    request = """
<webthumb>
    <apikey>%s</apikey>
    <request>
        <url>%s</url>
    </request>
</webthumb>
    """ % (WEBTHUMB_APIKEY, url)

    h = httplib.HTTPConnection(WEBTHUMB_HOST)
    h.request("GET", WEBTHUMB_URI, request)
    response = h.getresponse()

    type = response.getheader('Content-Type', 'text/plain')
    body = response.read()
    h.close()
    if type == 'text/xml':
        # This is defined as 'success' by the API. text/plain is failure.
        doc = xml.dom.minidom.parseString(body)

        for node in doc.getElementsByTagName("job"):
            wait = node.getAttribute('estimate')
            key = ""
            for node2 in node.childNodes:
                if node2.nodeType == Node.TEXT_NODE:
                    key = node2.data

        # We're given an approx time by the webthumb server,
        # we shouldn't request the thumbnail again within this
        # time.
        time.sleep(int(wait))

        request = """
    <webthumb>
        <apikey>%s</apikey>
        <fetch>
            <job>%s</job>
            <size>%s</size>
        </fetch>
    </webthumb>
        """ % (WEBTHUMB_APIKEY, key, size)

        h = httplib.HTTPConnection(WEBTHUMB_HOST)
        h.request("GET", WEBTHUMB_URI, request)
        response = h.getresponse()
        try:
            os.unlink(output_path)
        except:
            pass
        img = file(output_path, "wb")
        img.write(response.read())
        img.close()
        h.close()
        return True
    else:
        return False

From another Python script you can now create thumbs with ease:

from webthumb import get_thumbnail
get_thumbnail('http://www.rossp.org/', '/var/www/static/screenshots/rossp_org.jpg')

The error-checking is almost non-existant but so far it's worked for my purposes - I'm using it from within a cron job to fetch screenshots for new websites listed on a website.

Enjoy :)

Wedding Registry Post Mortem

I married by lovely wife on March 31st of this year. To handle our gift registry, I used my new Registry website (which I've spoken about in the past). Although I'm not quite ready to go live, I've analysed some of the data from our usage of the Registry system and made some interesting observations.

The chart to the right shows cumulative purchases over time, from the first purchase (around two months before the wedding, when we gave invites & registry details to our families) to the last purchase, 2 days before the wedding.

I've removed the numbers from the y-axis as they're irrelevant to everybody except me, but the axis hasn't been altered at all -- the scale has been kept intact and increases proportionately from 0 to n.

As you can see, the bulk of the purchases happened two-three weeks before the wedding. In fact, until 17 days before the wedding there were only around 10% of purchases.

Other quick observations that we noticed:

  • Although we listed our items in order of priority, the priority not the most important factor to purchasers (this takes into account price of items too -- we didn't just put big-ticket items at the top of the list, it was (we belive) a pretty even spread of prices.
  • Most people veered away from suggestions that were unique or otherwise not mainstream. This is probably a valid representation of our invite list, more than anything -- we did get some fantastic unique gifts from people who are more open to being untraditional!
  • More traditional guests (mostly our grandparents or older relatives) still prefer traditional gifts and/or cash. Nothing will change that, and nor should it change.
  • Only one guest didn't understand the registry system -- they thought they were shopping online rather than selecting gifts they can then go and buy themselves. I've since clarified the wording on the website.
  • Feedback shows guests were happy to be able to browse a list of items we had suggested for them to buy us, and they were happy to then be able to go to their own choice of store and make the purchase.
  • A few guests were able to use their work or other industry contacts to get us items off the list that we wanted, without spending as much of their hard-earned cash. This is the way I planned for the registry website to be used!

For most people this is probably relatively useless information -- but we found it interesting and it may assist others with preparing their registries.

I'm hoping to announce my registry website soon so people can start using it -- I'm moving it to a new server at the moment, and changing the look and feel to be a little less nerdy. Stay tuned.

Using Subdomains with Django

As a part of my previously mentioned upcoming bridal gift registry project (which, by the way, performed outstandingly in it's most important private beta ever - my own wedding) I'm giving each user (in this sense, a user is a couple close to getting married) their own subdomain off of the main website - instead of having a URL to their registry like http://yourdomain.com/registries/view/?id=1048 there are beautiful URL's like http://couplesnames.yourdomain.com.

Getting this working was really quite simple, and it's something that I can see being useful for SAAS projects - just take a look at the 37signals projects such as Basecamp and the way they handle per-company logins: each company has it's own subdomain.

The first thing that needs to be done is server configuration. Your domain name needs to have a wildcard entry setup (eg *.yourdomain.com), using Bind9 this is as easy as adding the following line to your domain name config. For other setups, speak to your web hosting provider.

*       CNAME   yourdomain.com.

Next, configure your web server software to answer requests for *.yourdomain.com the same way as www.yourdomain.com. I'm assuming that www.yourdomain.com already works, and that you're using Apache - again, YMMV so speak to your hosting provider. In Apache, just add the following line to your yourdomain.com virtual host configuration:

ServerAlias *.yourdomain.com

After restarting your DNS & web server software, you'll find that going to http://anything.yourdomain.com shows up the same result as http://www.yourdomain.com. Perfect.

Lastly you need to configure Django to treat subdomains differently. In this case I'm making another assumption, that Django is handling requests for yourdomain.com, and that there is a view already in place (eg via urls.py) for handling requests for the root of the domain. I'm also assuming content for your "main page" (eg, anything that's not a subdomain) is handled by other views, and that your main page is under /mypage/.

My 'index' view therefore looks like this:

def index(request):
    django_site = Site.objects.get_current()
    if request.META['HTTP_HOST'] == django_site.domain:
        # The visitor has hit the main webpage, so redirect to /mypage/ 
        return HttpResponseRedirect('/mypage/')

    # Split the domain into it's parts, remove the main part of the domain 
    # from the requested host, and we're left with one variable: 'subdomain'.
    # We also strip out the 'www.' non web-savvy users often type 'www' in 
    # front of every website they visit, so let's not show them an error message!
    domain_parts = django_site.domain.split(".")
    domain = ".".join(domain_parts[1:])
    subdomain = request.META['HTTP_HOST'].replace(domain, '').replace('.', '').replace('www', '')
    # You can now access your data models using 'subdomain', eg:
    couple = Couple.objects.get(subdomain=subdomain)

You can see I'm referring to a model called Couple, with a field named 'Subdomain'. All you need to do is refer to your own model with a 'subdomain' key and you're in business - just remember to enforce unique subdomains at signup time! Lastly, ensure your current Django 'Site' entry (django.contrib.sites) is up to date with the domain set to 'www.yourdomain.com'.

From here you can customise what's shown to the visitor based off of the subdomain they've hit. Very easy to setup, and potentially very powerful - especially in environments where people need something as easy as possible to remember or as unique as possible.

Helping Sort Django's Error Emails

When you run a Django powered website with debugging turned off, whenever a condition happens that would cause a HTTP 500 Error (such as an uncaught exception), the administrators receive an e-mail with a traceback and other relevant details to help debug the issue. These e-mails are really, really handy.

If however you run multiple Django sites, all with Debugging turned off, it can become difficult to tell which emails and errors relate to which site.

There are two solutions:

  1. Write perfect code that never throws errors; or
  2. Get Django to prefix the subject of the error e-mails so you can tell what belongs where.

Since there are only 24 hours in a day (most of which when I'm either sleeping or at my non-Django day job which pays the bills) I've opted for #2. And doing that is easy.

All you need to do is set the EMAIL_SUBJECT_PREFIX variable in settings.py:

EMAIL_SUBJECT_PREFIX = '[myproject] '

Easy as pie!

What's Wrong with Bridal Registries

Next month, I'm getting married. For reasons unknown to me, getting married costs money. Lots of money.

That's not a huge problem, and to be honest it's not overly surprising. It just strikes me as odd that an entire industry is built around charging exhorbitant amounts of cash for services identical to those offered to others at much lower prices - and people pay for it.

Yet it's not just the bride and groom getting ripped off when it comes to the big day - wedding guests are expected to buy gifts for the happily married couple (which, of course, I don't have an issue with) however these same guests are expected to purchase their goods from overpriced department stores from pre-selected gift lists.

Why, as a couple getting married, should we lock our guests (who are very generous in deciding to buy us something to assist with our new life together) into being ripped off when they buy us gifts?

This portion of the industry needs a change - and I'm working hard on it. I'm starting a new service for couples getting hitched, to let them give everybody more flexibility when it comes to bridal registries.

Couples will have more flexibility when it comes to selecting gifts they'd like to receive. Guests will have more flexibility when it comes to purchasing gifts. Most importantly, nobody will be ripped off.

My new service, which I'll be launching publicly soon, works. It works well. Hand-picked couples have used it over the past few months with fantastic feedback. I'm using it now for my wedding, again with fantastic feedback from guests.

Soon, you'll be able to use it too. Lets get some simplicity back into weddings.

Blog Spam Protection

I currently manually review and delete comment spam every few days. It doesn't take long, it's just frustrating.

When you regularly repeat an action over a long period of time, you begin to notice patterns of action. What I've noticed is that most comment spam is being posted to older blog posts - most likely due to their higher number of in-links, and I'm guessing higher google page rank.

To this end, what I've done is add a method to my blog Post model, which checks the post date and, if it's within the defined timeframe, will return 'True'. My Post detail template now checks this method and only displays the form when appropriate.

The first file to change is your blog applications models.py. At the top of the file add this import statement to bring in the 'datetime' module and required tools:

from datetime import datetime, timedelta

Next, in the same file, add the following method to your 'Post' class (just after the 'body' field is fine):

    def allow_comments(self):
        return (self.date + timedelta(21)) > datetime.now()

Lastly, open up your blog detail template and replace the existing call to {% freecomment_form ... %}_ with:

{% if object.allow_comments %}
{% free_comment_form for blog.post object.id %}
{% else %}
<p>Comments can only be posted for recent articles.</p>
{% endif %}

What this does is pretty straightforward. Using datetime and timedelta objects, we ensure that only posts within the last 21 days can have comments posted to them. For me, this timespan works well, you may need to experiment with longer or shorter times.

Next up for me is actually altering the 'comments' mechanism to add filtering via Akismet or similar processes - but I'm wary of modifying the contrib code as I dread upgrade-time!

More...

Want to see more? Check out the yearly archives below.