Tag Archive: geeky.

Self-Hosted Server Status Page with Uptime Robot, S3, and Upscuits

For quite a while I've had a public "Status" page online for WhisperGifts via Pingdom. It basically just shows uptime over the past few days, but given my site is relatively low-volume and not ovely critical to my customers, the $10/month for Pingdom was actually one of my largest expenses after hosting.

So, I started looking for an alternative.

Today I re-deployed the WhisperGifts Status Page using a combination of Uptime Robot, Upscuits and Amazon S3.

In short, I have Uptime Robot checking the uptime of my site (including it's subsites, such as the admin and user pages). The statistics are gathered and presented by Upscuits, which is entirely client-side JavaScript hosted on S3.

My basic todo list for next time:

  1. Sign up for Uptime Robot. I'd been using them for ages on their Free plan as a backup to Pingdom; this gives 5-minute checks. Their paid plan gives 1-minute resolution.
  2. Add your sites, make sure they're being monitored correct.
  3. On the Uptime Robot dashboard, click My Settings. Open the section labelled Monitor-Specific API Keys and search for your Monitor. Copy the API key to a text file for later; repeat this step for subsequent monitors you want to include on your status page.
  4. Download the latest Upscuits release to your PC.
  5. In the public folder of the upscuits package, rename config.example.js to config.js. Paste your API key(s) inside it.
  6. Create an AWS bucket called eg status.mysite.com and enable website mode. Setup your DNS etc to point to this bucket.
  7. Upload the contents of public/ to your AWS bucket
  8. Visit your new status page and view your last 12 months of Uptime Robot statistics
  9. Close your Pingdom account saving $10 a month Profit!

For a small site like mine this has a couple of obvious benefits. It's free (or $4.50/month if you want higher resolution - still half the price of the most basic Pingdom plan); it uses a tiny amount of S3 storage which is as good as free, and doesn't involve running any server-side code. The included index.html is also easily customisable if you like, since it's just plain HTML (using the Bootstrap framework, by default). This is a big win over hosted solutions, IMO.

How I use HuffDuffer as "Instapaper for Audio"

I like to listen to podcasts. There's the big names I listen to such as This American Life, their spinoff Serial, and Radiolab - but I also like listening to ad-hoc pieces of audio that I come across without subscribing to a whole podcast feed.

For a while now I've been doing this with Huffduffer. Huffduffer lets me have my own personal "podcast" feed made of individual audio files. In a way it's like Instapaper (another favourite of mine for collecting articles to read) but for listening.

I use the official Huffduffer Chrome extension. Whenever I come across a page with a link to an MP3 I want to listen to later, I hit the little Huffduffer 'hand' and the details are saved to my Huffduffer feed.

In my Podcast player (I use and like Overcast) I manually subscribe to my feed. It's at https://huffduffer.com/username/rss. Then, whenever I 'Huffduff' some audio, it appears as a podcast episode in Overcast.

A few places I've used this recently:

I've no idea if this is how Huffduffer is intended to be used, but it's bloody useful.

Jutda: Django-powered Solution Provider

In my recent catchup blog post I mentioned in passing a few projects including Jutda and WhisperGifts. Now, I'd like to formally introduce the former of these (with the latter coming very soon now)

When I'm not 'at work', I spend a significant amount of time working on other projects both independently and together with other talented developers. These days, all of these projects are powered by Django.

Jutda is the glue that will pull these projects together under a single name.

The name comes from the nearly-extinct Wagiman language, spoken by a small number of indigenous Australians. It means "to point" or "to show", which I've taken on as the ethos of my new company: We’re aiming to show the way to others, to make life easier through the use of elegant online solutions.

The ultimate aim is to create simple solutions for everyday problems. No bells, no whistles, just beautiful outcomes for ugly problems.

More information about Jutda can be seen on the About Jutda page, and in the initial blog post I published on the Jutda website.

The first project that's been finished and published is WhisperGifts, about which I'll write more in a future blog post. I'm very excited about the possibilities of this, and the impact it may have on couples in the process of organising their wedding festivities.

I've also got a few other projects in the pipeline, with no fixed feature-set or timeline. Projects that are well on their way to adulthood include:

  • A personal fixed asset register: Do you know the serial number of your television? The warranty details for your refrigerator? The phone number of the guy who borrowed your Playstation for his son to use over Christmas? What would you do if your house burnt down tomorrow, taking all of your warranty paperwork with it? I'm helping you sort out this mess.
  • A personal information manager: There's been many takes on these, but none of them suit my needs quite well enough. Why can't I keep track of pages of information along with photos, embedded maps, lists, enhanced details of my contacts, and more - all in one place?
  • An open-source ticket tracker / helpdesk system. We're currently using a Django-powered helpdesk system internally to manage WhisperGifts support queries, and we're in the process of cleaning it up and abstracting it out from our other systems so that we can publish it for everybody to see and use. It's written in Django, it has full e-mail integration, it has both public and private web interfaces, and it provides open access to your data. It rocks.

So that's Jutda. Although I'm not planning for it to take over and become a full-time job, I hope to get some useful tools out in the wild for everyday consumers, whilst contributing further code back to the Django community wherever possible. Wish me luck!

Overdue Catchup

I've had a very busy few months in every way conceivable - everything from my Django projects, to my day job, to life as a whole has been running in fast-forward. Here's a quick summary of the

DjangoSites is coming along very well, with 1040 sites listed as of this evening. The quantity and quality is ever-increasing, and more and more sites are being claimed. There are still well over 300 unclaimed sites - is yours listed there? If so, drop an email with your DjangoSites username to djangosites@djangosites.org.

After a server move late last year my Django OpenID project went offline for a little while. After a handful of requests from the blogosphere I've put it back online - see my original blog posting on the topic for more details, although I'm guessing that Simon Willison has something up his sleeve that'll trump my hack-job soon enougy.

Over my Christmas holidays I launched Jutda, the 'corporate' face for my upcoming web projects built with Django. The word Jutda is from the Wagiman language, a dialect spoken by an ever-shrinking Aboriginal tribe in the Northern Territory of Australia. It means show the way, which is something I hope to do with my projects. This is by no means my day job, rather a single name with which to pull together a number of after-hours projects. Hopefully the name starts to mean something within the Django community after a little while :)

The first project to be released by Jutda is WhisperGifts, a service to allow you to publish your wedding gift registry online with minimum fuss. I used it for my wedding almost a year ago with no problems, and it's been used by others before and since with great praise from happily married couples and their guests alike. Of course, it's all built with Django, with a whole host of neat features. When I have a spare evening I'll write up a few more details, but in the mean time check it out and let me know what you think.

Last but not least, I thought I'd bring attention to a simple Django application I wrote a while ago but never 'released' per-se. Django-forum is a simple Forum application for Django, allowing you to leverage your existing templates and user accounts to add discussion-forum capability to your existing project. At least a few people are using it, with a few patches coming from the community to add new functionality. I don't currently have any public sites running it, but keep your eyes peeled.

That's all of my news for now - I've got plenty more to share, however just a little more time is needed to give a few projects some more polish before I go public. Let's just hope my next update isn't another three-months away!

Django Menuing System

On most of the websites that I've built with Django, I have had a desire to be able to manage little elements of the website from the Django administration screen without having to touch my templates. My intent is for the templates to become the presentation vehicle, with anything that matters being built out of the Django databases.

One such thing that I want to keep out of my templates is navigation. Sure, the template has a place for navigation (including an empty

    ), but the contents of my navigation bars are driven by a dynamic Django application.

    The application has but two files: the models and a template tag.

    First up is the model file, menu/models.py:

    from django.db import models
    
    class Menu(models.Model):
        name = models.CharField(maxlength=100)
        slug = models.SlugField()
        base_url = models.CharField(maxlength=100, blank=True, null=True)
        description = models.TextField(blank=True, null=True)
    
        class Admin:
            pass
    
        def __unicode__(self):
            return "%s" % self.name
    
        def save(self):
            """
            Re-order all items at from 10 upwards, at intervals of 10.
            This makes it easy to insert new items in the middle of 
            existing items without having to manually shuffle 
            them all around.
            """
            super(Menu, self).save()
    
            current = 10
            for item in MenuItem.objects.filter(menu=self).order_by('order'):
                item.order = current
                item.save()
                current += 10
    
    class MenuItem(models.Model):
        menu = models.ForeignKey(Menu)
        order = models.IntegerField()
        link_url = models.CharField(maxlength=100, help_text='URL or URI to the content, eg /about/ or http://foo.com/')
        title = models.CharField(maxlength=100)
        login_required = models.BooleanField(blank=True, null=True)
    
        class Admin:
            pass
    
        def __unicode__(self):
            return "%s %s. %s" % (self.menu.slug, self.order, self.title)
    

    Next is a template tag that builds named or path-based menus - menu/templatetags/menubuilder.py:

    from menu.models import Menu, MenuItem
    from django import template
    
    register = template.Library()
    
    def build_menu(parser, token):
        """
        {% menu menu_name %}
        """
        try:
            tag_name, menu_name = token.split_contents()
        except:
            raise template.TemplateSyntaxError, "%r tag requires exactly one argument" % token.contents.split()[0]
        return MenuObject(menu_name)
    
    class MenuObject(template.Node):
        def __init__(self, menu_name):
            self.menu_name = menu_name
    
        def render(self, context):
            current_path = template.resolve_variable('request.path', context)
            user = template.resolve_variable('request.user', context)
            context['menuitems'] = get_items(self.menu_name, current_path, user)
            return ''
    
    def build_sub_menu(parser, token):
        """
        {% submenu %}
        """
        return SubMenuObject()
    
    class SubMenuObject(template.Node):
        def __init__(self):
            pass
    
        def render(self, context):
            current_path = template.resolve_variable('request.path', context)
            user = template.resolve_variable('request.user', context)
            menu = False
            for m in Menu.objects.filter(base_url__isnull=False):
                if m.base_url and current_path.startswith(m.base_url):
                    menu = m
    
            if menu:
                context['submenu_items'] = get_items(menu.slug, current_path, user)
                context['submenu'] = menu
            else:
                context['submenu_items'] = context['submenu'] = None
            return ''
    
    def get_items(menu, current_path, user):
        menuitems = []
        for i in MenuItem.objects.filter(menu__slug=menu).order_by('order'):
            current = ( i.link_url != '/' and current_path.startswith(i.link_url)) or ( i.link_url == '/' and current_path == '/' )
            if not i.login_required or ( i.login_required and user.is_authenticated() ):
                menuitems.append({'url': i.link_url, 'title': i.title, 'current': current,})
        return menuitems
    
    register.tag('menu', build_menu)
    register.tag('submenu', build_sub_menu)
    

    Using this menu system is relatively easy:

    1. Filesystem setup

      1. Create a directory called menu in your Python path
      2. Drop models.py (above) into the menu folder
      3. Create a directory called templatetags inside the menu folder
      4. Copy the menuubilder.py (above) into the templatetags folder
      5. Create a blank file called _init.py_ and put a copy in each of your menu and templatetags folders
    2. Django setup

      1. Add menu to the INSTALLED_APPS list in your settings.py
      2. Run ./manage.py syncdb to create the relevant database tables
    3. Data setup (using Django's Admin tools)

      1. Add a Menu object for each menu set you wish to use. Give static menus (eg, those that are the same on each page) a slug such as main, footer or sidebar. For dynamic menus, that display different contents on different pages, add a base URL. A menu with a base URL or /about/ might contain links to your philosophy, your team photos, your history, and a few policies - but only when the user is visiting a page within /about/.
    4. Template setup

      1. In your template, wherever you want to use a particular named menu, add this code:

        {% load menubuilder %}{% menu main %} {% for item in menuitems %}
      • {{ item.title }}
      • {% endfor %}

      1. Replace main with the name of a static menu (eg footer or sidebar from the above example)
      2. For dynamic URL-based menus, add this code:

      {% load menubuilder %}{% submenu %}{% if submenu %}

      {% endif %}

      1. If a menu has been set up with a BASE URI that the user is currently seeing, it will be displayed. If no such menu exists, nothing will be displayed.

    I use named menus for link bars in header and footers. I use URI-based menus to display a sub-section navigation, for example within the about area on a website or within the products section. A URI-based menu will only be displayed with the user is within that URI.

    So there you have it. Database driven menus that let you build easy static or URI-based menus and submenus. If you look carefully there is code to only display links to logged in users if required, and the page that is currently being viewed is tagged with the current CSS class so it can be easily styled.

    If all of that is confusing, stay tuned for my Basic Django-powered website in a box that will pull together a number of elements such as this into a simple to install package you can use to get a website up and running quicksmart - with the fantasmical automated Admin screens to control it all.

DjangoPoweredSites Grows Up

After some e-mail discussions with Jacob Kaplan-Moss from the Django team, I've moved all of the sites listed on the old DjangoPoweredSites wiki page to Djangosites.org.

It took a fair bit of time to weed out some not-working pages and expired domains (which have, of course, been purchased by spammers and link farmers), and a small investment in extra WebThumb credits to allow me to take screenshots of a fresh 500+ websites, but I got there in the end.

We'll now be closing the DjangoPoweredSites wiki page for good, as it has become redundant and is not the best way to peruse 800+ websites.

What we're left with is a fantastic directory of websites powered by Django. I'll be putting together a 'Featured' list shortly, but in the meantime I need a little bit of help from the community.

All sites scraped from the wiki page are currently unowned. I encourage you to check out the list of unclaimed websites, and e-mail me to let me know which sites are yours so you can receive due credit. There are some great websites listed there, so it'd be fantastic to be able to show off who built them.

Thanks to Jacob and the rest of the Django team for getting behind this initiative.

Easy Multi-Part E-Mails with Django

Every time I send e-mail with Django I seem to do it slightly differently, especially when it comes to sending HTML-based emails.

So, I did what any good Djangonaut does and wrote a quick function that wraps up sending of a multi-part HTML / Text e-mail from templates you create.

Firstly, create your e-mail templates. You'll need both a HTML and a plain-text template; for the HTML template I strongly recommend the use of Premailer, which puts all CSS definitions inline.

Here's a sample HTML template - as it would come out of Premailer. We'll pretend it's in the emails template directory, with a name of newsite.html.

<h1 style='font: Arial; font-weight: bold; font-size: 15pt; color: #006;'>Thanks!</h1>

<p style='font: Arial,Helvetica; size: 10pt;'>Hello, {{ username }}.</p>

<p style='font: Arial,Helvetica; size: 10pt;'>Thank you for submitting your website, titled <strong>{{ title }}</strong>, to our listing.</p>

<p style='font: Arial,Helvetica; size: 10pt;'>We'll verify it, take a screenshot, and publish it within the next few days.</p>

<p style='font: Arial,Helvetica; size: 10pt;'>Regards,</p>

<p style='font: Arial,Helvetica; size: 10pt;'><em>Your Favourite Webmaster.</em></p>

And here's that same e-mail, in plain-text format - also in the e-mails folder, called newsite.txt.

Hello, {{ username }}.

Thank you for submitting your website, titled '{{ title }}', to our listing.

We'll verify it, take a screenshot, and publish it within the next few days.

Regards,

Your Favourite Webmaster.

Those who are familiar with Django's template language will see that this template uses two variables, username and title. Let's populate them, shall we?

template_context = {
    'username': 'joecitizen',
    'title': 'My Movie Review Site',
    'url': 'http://www.moviewreviews.dom',
}

So far we have an e-mail message template in both HTML and text formats; all we need now to send an e-mail is a subject and a recipient.

recipient = 'joe@example.dom' # ['joe@example.dom',] would also be suitable
subject = 'Site submission: {{ url }}'

That's pretty straightforward - our subject also contains Django template variables, in this case the URL from our template context we created earlier.

Now it's time to send the message. I have been using this function with much success:

def send_multipart_mail(template_name, email_context, subject, recipients, sender=None, fail_silently=False):
    """
    This function will send a multi-part e-mail with both HTML and
    Text parts.

    template_name must NOT contain an extension. Both HTML (.html) and TEXT
        (.txt) versions must exist, eg 'emails/public_submit' will use both
        public_submit.html and public_submit.txt.

    email_context should be a plain python dictionary. It is applied against
        both the email messages (templates) & the subject.

    subject can be plain text or a Django template string, eg:
        New Job: {{ job.id }} {{ job.title }}

    recipients can be either a string, eg 'a@b.com' or a list, eg:
        ['a@b.com', 'c@d.com']. Type conversion is done if needed.

    sender can be an e-mail, 'Name <email>' or None. If unspecified, the
        DEFAULT_FROM_EMAIL will be used.

    """
    from django.core.mail import EmailMultiAlternatives
    from django.template import loader, Context
    from django.conf import settings

    if not sender:
        sender = settings.DEFAULT_FROM_EMAIL

    context = Context(email_context)

    text_part = loader.get_template('%s.txt' % template_name).render(context)
    html_part = loader.get_template('%s.html' % template_name).render(context)
    subject_part = loader.get_template_from_string(subject).render(context)

    if type(recipients) != list:
        recipients = [recipients,]

    msg = EmailMultiAlternatives(subject_part, text_part, sender, recipients)
    msg.attach_alternative(html_part, "text/html")
    return msg.send(fail_silently)

Lastly, to put together the previous example:

from multipart_email import send_multipart_email # Assuming, of course, you saved the above functoin in multipart_email.py

send_multipart_mail('emails/newsite', template_context, subject, recipient)

What this does, is uses our existing template context, subject, and recipient, to send a HTML and plain-text e-mail based off the templates we built earlier. Note that it automatically appends .html or .txt as required. Hopefully the extra documentation in the docstring is useful.

Recipients who can read HTML e-mail will see your nicely formatted HTML e-mail message, while everyone else will see the plain-text alternative.

Slowly, this method is finding it's way into each of my Django applications. Please, just don't overdo the HTML messages!

Django: Multiple Aliases for a Single Website

In these days of cheap domains, it's often desirable to own multiple domains for a single website. You've probably got each of the .com, .net and .org domain names, along with a country-specific domain. You want each of these to present exactly the same website to the world, but good design says that each web page should have one, and exactly one, URL. So what's the best way to serve this up without having an Apache config for each domain?

I've come across this whilst building a website recently whereby the primary domain is mydomain.com.au, while I've got secondary domains in other popular TLD's to try and reduce domain squatting and the like.

One option is to configure an Apache virtual host for each domain, which serves up a static redirect. Another is to have Apache aliases for the main host, so each of the domains serves up the same content. This works, but leaves each page with multiple URL's.

My solution is to set up Apache aliases, and use a Django middleware to identify any requests that aren't for the main domain name, redirecting them as they're found. The middleware code I use is as follows:

from django.http import HttpResponsePermanentRedirect

class ValidateHostMiddleware(object):
    """
    Redirect all requests for a domain other than mysite.com.au
    """
    def process_request(self, request):
        if not request.META['HTTP_HOST'].endswith('mysite.com.au'):
            return HttpResponsePermanentRedirect('http://www.mysite.com.au%s' % request.path)

This is nice and simple, and a useful way of having multiple domains (possibly increasing your virtual 'geographical spread') but keeping your search-engine optimisation efforts intact.

Update: Thanks to a note from Brice Carpenter, I've updated the code to do a permanent HTTP redirect (code 301) rather than a temporary (302) redirect. I've also added code from my live environment that sends the visitor to their request path on the new domain - so hitting www.mysite.com/about/policies/ refers the user to www.mysite.com.au/about/policies/.

Django: A Diverse Community

Scott Barnham, one of the guys behind the recently-launched Django Gigs website, has posted some statistics from visitors to the Gigs website over the past few days. I've put together some similar stats for the Django Sites website, which has been online for a few months now.

Note: All figures are from Google Analytics, and percentages are rounded to make life easier.

Summary

Operating Systems

Windows 60%

Linux 21%

Macintosh 18%

Others 1%

The operating system split is very much skewed towards Windows, almost all of the Windows users on Windows XP. It's nice to see higher-than-average (compared to my other sites) usage of Linux & Macintosh.

Browsers

Firefox 66%

Internet Explorer 16%

Safari 7%

Opera 5%

Others 6%

No surprises here. Firefox figures are very similar to the Gigs site, with Safari & IE being a bit different. I'm guessing the higher usage of IE shows the target audience of DjangoSites - people who are looking for what Django is capable of, perhaps newer-comers to the Django world rather than long-term stable mates who may be more likely to use Firefox or Safari on their shiny Macbook Pro.

Geographical Areas

North America 30%

Western Europe 18%

Eastern Europe 9%

Southern Europe 9%

Northern Europe 9%

South America 6%

Eastern Asia 4%

Australia & New Zealand 3%

Elsewhere 12%

Again, this isn't really too surprising. There is a very long-tail (and I'm not overly sure Google Analytics needs to break down Europe & Asia into so many sub-continents), easily visible with "Elsewhere" covering 12% of the visits.

Stuff I Found Interesting

  • Bolivian visitors viewed 20 pages/visit, followed by 18/visit for Oman. Everyone else was miles behind this.
  • 52% of traffic is from referrals, most of it from the Django website, an article in Smashing Magazine, and Wikipedia.
  • 30% of traffic is search-engine generated, typically from searches for 'django sites' - and somehow, including a few searches for 'Auction'.

Hopefully these numbers are interesting to people looking to target their offerings to the Django audience. Do they match your sites?

Mixing OpenID into Django's authentication system

NOTE: This code is now outdated, and it's certainly not the best way to do OpenID in Django. I recommend you take a look at django-authopenid, a fantastic registration system that combines Django's authentication framework with OpenID sign-in. Ross, 17th April 2008

According to the OpenID website, from a consumers point of view OpenID is "the elimination of multiple user names and passwords and a smoother, more secure, online experience. ". What it provides is a single identity for you to use at multiple websites. Instead of having a username and password for each website you peruse, you have an identity (usually a URL to your blog or an OpenID provider) that you use to login. The only password you have to remember is that of your OpenID provider - and you don't have to provide your password to any websites you visit.

Simon Willison has been a fantastic campaigner for OpenID, especially for integration with Django. He's written the fantastic django-openidconsumer package which provides the framework for a Django Application to act as an OpenID consumer (that is, people login to your Django app using their OpenID).

Simon's package creates a new OpenID object within your application, but is unrelated to the existing authentication system. So what I've done is mixed some glue to pull together these fantastic standalone applications:

The 'glue' provides a useful process flow for new users to your website. It lets people register with a username/password as they would have before OpenID, lets people login with OpenID's, and keeps it all intertwined. The basic functions are as follows:

  • A new user can sign up with a username/password, using django-registration
  • A new user can enter an OpenID, which is authenticated before creating a standard Django user. The account is verified using django-registration's email checker.
  • A user who is already logged in with a username/password, can login again using an OpenID, which is automatically associated with their username
  • A user can login with either their username/password (if they have one), or with their OpenID(s) (if they have any), and you'll always see them as 'request.user' - the same as if they had signed up with a username/password
  • This gives users a choice when they sign up and lets them change their mind by adding or removing OpenID's from their account as they wish.
  • Why do e-mail verification when users sign up with an OpenID? Because OpenID doesn't guarantee a users identity. It's merely a replacement for a username & password. E-Mail verification helps cut down on automated registrations, and ensures users are providing you with a valid contact address.

How would I use this in the real world? For sign-ups, I would show one form, clearly divided, asking a user to enter their OpenID if they have one, or otherwise enter their username/password/e-mail address. For logins, show users a username/password box with a link to toggle an OpenID entry field. If the user logs in with an OpenID, set a cookie so you can remember them in the future, showing them the OpenID login by default rather than the Username-driven login. Make it as easy as possible for people to adopt this fantastic technology.

I'm not using this code in a live environment yet, as it needs more testing. However, I've put a testbed online and made the source code available.

I would love it if you could try out this code both on my server and on yours, and provide feedback on the flow from a users point of view. I plan on cleaning up the code significantly before packaging it into something that is safe to use in a production environment.

The online demo is at http://openid.rossp.org. There is a link to the source code there.

Note: The database isn't accessible via the web, and it's a database set up specifically for this demonstration. I will NOT be extracting lists of usernames, passwords, e-mail addresses, openid's, etc. I may pu tup some usage statistics later, but certainly nothing identifiable. Please feel safe using this.

Validating a Username via jQuery with Ajax

It all starts when John hits your website and clicks the big 'Register' link. John types his name, '_John_' into the username box, and hands over his e-mail address and password (unless you're cool and hip, and you let him sign up using his OpenID) and hit 'Submit', just like every other website he's signed up to in the past.

Except this time, somebody else called John (what are the chances, eh?) has already signed up using that username, so after waiting a few seconds John sees an error message asking him to select another username. He types a new username and tries 'Submit' again, unsure as to whether his new selection will be suitable. So we fix this problem easily - we tell your users, while they're entering their username, whether their selection is available.

To achieve this we're going to use jQuery with it's fantastic Ajax support.

To get started, we create a simple view in Django that confirms the presence of a given username. This view will be accessible at /check_username/; and will expect a username via a GET paramater.

def checkusername(request):
    from django.contrib.auth.models import User
    from django.http import HttpResponse
    username = request.POST.get('username', False)
    if username:
        u = User.objects.filter(username=username).count()
        if u != 0:
            res = "Already In Use"
        else:
            res = "OK"
    else:
        res = ""

    return HttpResponse('%s' % res)

You'll now find that accessing /checkusername/?username=john_ will return 'Already in Use' or 'OK', as required.

The next thing to do is access it from within your registration form. I use James Bennett's fantastic django-registration with a customised template. This means I don't have to alter the registration code at all! I've added a little element just next to the username field, which we'll update shortly. The username field on my form looks like this:

<dt><label for="id_username">Username:</label></dt>
<dd>{{ form.username }} <span id='username_status'></span> {% if form.username.errors %}<span class="formerror">{{ form.username.errors|join:", " }}</span>{% endif %}</dd>

We've got an empty there called usernamestatus_. Now, when the user types a username, we want to check that username via background AJAX call and update the <span%gt; appropriately.

In the header of your registration form (I use a block in my templates called {% block extrahead %} which is inside the of my base.html) you will need to add a handful of lines of JavaScript:

<script type='text/javascript' src='/media/jquery.js'></script>
<script type='text/javascript'>
var previous_username = '';
var in_ajax = 0;
function checkUsername() {
    username = $("#id_username").val();
    if ((previous_username != username) && (username != '') && (in_ajax != 1)) {
        in_ajax = 1;
        $("#username_status").html("<img src='/media/busy.gif' />");
        $("#username_status").load('/check_username/', {username: username}, function() {in_ajax = 0;});
    }
    previous_username = username;
}
$(function() {
    setInterval("checkUsername()", 1000);
});
</script>

This code is relatively simple. Firstly, it loads jquery.js, which I hope by now you've downloaded and placed into a suitable media directory. Then, using the last 3 lines, it causes the checkUsername() function to be called every second. This function does an AJAX request to our checkusername_ view and puts the response into our usernamestatus_ .

We do a few niceties here, too. Firstly, we make sure the username isn't the same as when we last checked it (no point in checking the same username every second - it would increase server load with no real benefit). We also put a 'busy' image into the status area whilst doing the AJAX call - I use a simple image from ajaxload.info.

Net result? The user gets real-time feedback on whether their username is available, so they don't have to think as hard when the page re-loads with an error message.

Note that this won't stop the user submitting a form with an invalid username - if they do submit it, they'll get an error message as per usual. All this function does is provide some nice feedback to users who wish to use it. Signing up to a website should be easy, after all.

DjangoSites: We Want YOU!

I've just approved another batch of sites bringing the total to 260. I've also had to delete a few sites that were submitted due to a number of reasons: A handful were submitted that are obviously not Django (providing your link as /index.php is a give away) and a few have been submitted that are inaccessible URL's.

If you've got a website built with Django, we'd love for you to submit it. There are about 600-700 sites listed on the wiki page, if we can get most of those moved over to DjangoSites.org it'd be fantastic. It's a great way to show off what's out there in the wild that's powered by Django, and a great way to get feedback on your sites.

For those learning Django, it's also a great way to see how others have built their sites. Our listing of sites with publicly available source is growing every week, and is a great way to view how different problems have been tackled.

In the meantime, maddiin and I are working on a repository for Django-powered applications. There are plenty of projects out there (especially on Google Code), however having listed together in one place can only be a good thing. At a glance, you'll be able to see what version of Django and Python you need for a given application, and you can set your user preferences to only show applications that run on your version of Django so you aren't constantly looking at potentially incompatible applications. Keep your eyes here for more upcoming information about this.

Djangosites Updates

Last week I launched djangosites.org, a new website to show off other websites built with Django. Uptake has been great (as I write this there are 125 websites listed) and feedback has been even better.

Firstly there are more RSS feeds. You can view RSS feeds for latest entries, particular tags (eg business), or individual authors. The 'RSS' button at the top-right of most pages links to an RSS feed of your current view, more will be coming soon.

Next, listings with source-code available are now more easily identifiable from the listings by a new icon: (a good example of this is the blog tag listing which has a few listings with source-code available)

We've also done a fair few minor cosmetic changes, eg signup emails now come from a sane email address, and the signup/login forms match the rest of the site

Any other suggestions are more than welcome, in the meantime maddiin and I are working on our next Django community project! Stay tuned for more details.

Introducing Djangosites.org

Until now, if you wanted to see a few sites that were built with Django you had to wade through a list of a few hundred links on a wiki page. The links were somewhat organised into categories, but no matter who you ask that's a boring job.

Djangosites.org has been built as a showcase of what's out there in Django Land.

We hope it lets you easily see what's capable with the Django framework. Feel free to add your sites, post comments on other sites, and vote for your favorites. What I'd really love is if some of the Django heavyweights could post some sites up - it would kick ass if this can eventually replace the DjangoPoweredsites wiki page.

Thanks to maddiin for the site design, and to the django-users mailing list for feedback and suggestions. If you have any feedback please don't hesitate to email it to me at ross at this domain.

Using The WebThumb API with Python

First things first: Get an API key from bluga.net by signing up for an account.

Next, drop the following code into webthumb.py and enter your API key.

"""
Python interface to Webthumb API (see http://bluga.net/webthumb/)

By Ross Poulton - www.rossp.org

License: Use this how you like, just don't claim it as your own because
         that isn't cool. I'm not responsible for what this script does.

Usage: Define WEBTHUMB_APIKEY with your API key, as per the above URL.

Then, just call get_thumbnail(url, output_path). It will return true on
success, false on anything else.

An optional third parameter can be passed for the image size.
"""

import time
import os
import httplib

import xml.dom.minidom
from xml.dom.minidom import Node

WEBTHUMB_APIKEY='Enter your webthumb API key here'

WEBTHUMB_HOST='webthumb.bluga.net'
WEBTHUMB_URI='/api.php'

VALID_SIZES = (
    'small',
    'medium',
    'medium2',
    'large',
)

def get_thumbnail(url, output_path, size='medium2'):
    if size not in VALID_SIZES:
        return False

    request = """
<webthumb>
    <apikey>%s</apikey>
    <request>
        <url>%s</url>
    </request>
</webthumb>
    """ % (WEBTHUMB_APIKEY, url)

    h = httplib.HTTPConnection(WEBTHUMB_HOST)
    h.request("GET", WEBTHUMB_URI, request)
    response = h.getresponse()

    type = response.getheader('Content-Type', 'text/plain')
    body = response.read()
    h.close()
    if type == 'text/xml':
        # This is defined as 'success' by the API. text/plain is failure.
        doc = xml.dom.minidom.parseString(body)

        for node in doc.getElementsByTagName("job"):
            wait = node.getAttribute('estimate')
            key = ""
            for node2 in node.childNodes:
                if node2.nodeType == Node.TEXT_NODE:
                    key = node2.data

        # We're given an approx time by the webthumb server,
        # we shouldn't request the thumbnail again within this
        # time.
        time.sleep(int(wait))

        request = """
    <webthumb>
        <apikey>%s</apikey>
        <fetch>
            <job>%s</job>
            <size>%s</size>
        </fetch>
    </webthumb>
        """ % (WEBTHUMB_APIKEY, key, size)

        h = httplib.HTTPConnection(WEBTHUMB_HOST)
        h.request("GET", WEBTHUMB_URI, request)
        response = h.getresponse()
        try:
            os.unlink(output_path)
        except:
            pass
        img = file(output_path, "wb")
        img.write(response.read())
        img.close()
        h.close()
        return True
    else:
        return False

From another Python script you can now create thumbs with ease:

from webthumb import get_thumbnail
get_thumbnail('http://www.rossp.org/', '/var/www/static/screenshots/rossp_org.jpg')

The error-checking is almost non-existant but so far it's worked for my purposes - I'm using it from within a cron job to fetch screenshots for new websites listed on a website.

Enjoy :)

Using Subdomains with Django

As a part of my previously mentioned upcoming bridal gift registry project (which, by the way, performed outstandingly in it's most important private beta ever - my own wedding) I'm giving each user (in this sense, a user is a couple close to getting married) their own subdomain off of the main website - instead of having a URL to their registry like http://yourdomain.com/registries/view/?id=1048 there are beautiful URL's like http://couplesnames.yourdomain.com.

Getting this working was really quite simple, and it's something that I can see being useful for SAAS projects - just take a look at the 37signals projects such as Basecamp and the way they handle per-company logins: each company has it's own subdomain.

The first thing that needs to be done is server configuration. Your domain name needs to have a wildcard entry setup (eg *.yourdomain.com), using Bind9 this is as easy as adding the following line to your domain name config. For other setups, speak to your web hosting provider.

*       CNAME   yourdomain.com.

Next, configure your web server software to answer requests for *.yourdomain.com the same way as www.yourdomain.com. I'm assuming that www.yourdomain.com already works, and that you're using Apache - again, YMMV so speak to your hosting provider. In Apache, just add the following line to your yourdomain.com virtual host configuration:

ServerAlias *.yourdomain.com

After restarting your DNS & web server software, you'll find that going to http://anything.yourdomain.com shows up the same result as http://www.yourdomain.com. Perfect.

Lastly you need to configure Django to treat subdomains differently. In this case I'm making another assumption, that Django is handling requests for yourdomain.com, and that there is a view already in place (eg via urls.py) for handling requests for the root of the domain. I'm also assuming content for your "main page" (eg, anything that's not a subdomain) is handled by other views, and that your main page is under /mypage/.

My 'index' view therefore looks like this:

def index(request):
    django_site = Site.objects.get_current()
    if request.META['HTTP_HOST'] == django_site.domain:
        # The visitor has hit the main webpage, so redirect to /mypage/ 
        return HttpResponseRedirect('/mypage/')

    # Split the domain into it's parts, remove the main part of the domain 
    # from the requested host, and we're left with one variable: 'subdomain'.
    # We also strip out the 'www.' non web-savvy users often type 'www' in 
    # front of every website they visit, so let's not show them an error message!
    domain_parts = django_site.domain.split(".")
    domain = ".".join(domain_parts[1:])
    subdomain = request.META['HTTP_HOST'].replace(domain, '').replace('.', '').replace('www', '')
    # You can now access your data models using 'subdomain', eg:
    couple = Couple.objects.get(subdomain=subdomain)

You can see I'm referring to a model called Couple, with a field named 'Subdomain'. All you need to do is refer to your own model with a 'subdomain' key and you're in business - just remember to enforce unique subdomains at signup time! Lastly, ensure your current Django 'Site' entry (django.contrib.sites) is up to date with the domain set to 'www.yourdomain.com'.

From here you can customise what's shown to the visitor based off of the subdomain they've hit. Very easy to setup, and potentially very powerful - especially in environments where people need something as easy as possible to remember or as unique as possible.

Helping Sort Django's Error Emails

When you run a Django powered website with debugging turned off, whenever a condition happens that would cause a HTTP 500 Error (such as an uncaught exception), the administrators receive an e-mail with a traceback and other relevant details to help debug the issue. These e-mails are really, really handy.

If however you run multiple Django sites, all with Debugging turned off, it can become difficult to tell which emails and errors relate to which site.

There are two solutions:

  1. Write perfect code that never throws errors; or
  2. Get Django to prefix the subject of the error e-mails so you can tell what belongs where.

Since there are only 24 hours in a day (most of which when I'm either sleeping or at my non-Django day job which pays the bills) I've opted for #2. And doing that is easy.

All you need to do is set the EMAIL_SUBJECT_PREFIX variable in settings.py:

EMAIL_SUBJECT_PREFIX = '[myproject] '

Easy as pie!

Blog Spam Protection

I currently manually review and delete comment spam every few days. It doesn't take long, it's just frustrating.

When you regularly repeat an action over a long period of time, you begin to notice patterns of action. What I've noticed is that most comment spam is being posted to older blog posts - most likely due to their higher number of in-links, and I'm guessing higher google page rank.

To this end, what I've done is add a method to my blog Post model, which checks the post date and, if it's within the defined timeframe, will return 'True'. My Post detail template now checks this method and only displays the form when appropriate.

The first file to change is your blog applications models.py. At the top of the file add this import statement to bring in the 'datetime' module and required tools:

from datetime import datetime, timedelta

Next, in the same file, add the following method to your 'Post' class (just after the 'body' field is fine):

    def allow_comments(self):
        return (self.date + timedelta(21)) > datetime.now()

Lastly, open up your blog detail template and replace the existing call to {% freecomment_form ... %}_ with:

{% if object.allow_comments %}
{% free_comment_form for blog.post object.id %}
{% else %}
<p>Comments can only be posted for recent articles.</p>
{% endif %}

What this does is pretty straightforward. Using datetime and timedelta objects, we ensure that only posts within the last 21 days can have comments posted to them. For me, this timespan works well, you may need to experiment with longer or shorter times.

Next up for me is actually altering the 'comments' mechanism to add filtering via Akismet or similar processes - but I'm wary of modifying the contrib code as I dread upgrade-time!

Simple Javascript Spell Checking

The solution I chose in the end was Speller Pages, a very simple javascript-driven user interface with a number of different server-side options, each using the GNU Aspell application to do the actual spell checking.

Installation is extremely simple. Download the archive, place it on your web server (under /speller/ seems to be the quickest to get going), edit the JavaScript file to point it at either a Cold Fusion, PHP or Perl back-end, and add some extremely simple JavaScript calls to the page where you want the spell check to appear.

From the users point of view, all they have to do is click the obvious "Check Spelling" button next to the relevant textarea, and a logical, simple to use window appears highlighting the incorrect spellings and providing a list of relevant alternatives. Once they're finished selecting replacement words, the window automatically disappears and the user continues on their merry way - without having to worry about the embarrassment of sending their friends and family to a website full of typos or misspellings.

Total integration time, including making a new spell-check button on the page (with a bit of assistance from the icons by Mark James) , was about fifteen minutes. Not bad for a piece of functionality that makes life just that bit easier to use.

The Pareto principle (Or, why the 80/20 rule always bites us in the ass)

This rant bought to you by yet another unfinished project.

As hard as you work to avoid it, there seems to be no way to avoid the good old 80/20 rule. It seems that no matter what you're working on, 80% of the work takes 20% of the time, and the remaining 20% of the work takes (at least) 80% of the remaining time on the project.

The '80/20 rule', as it's commonly known, is an implentation of the Pareto Principle, originally conceived by observing that 80% of income in italy was received by 20% of the Italian population. (Source: Wikipedia).

In software development this is very visible. It doesn't take a developer long to get a working mock-up online, complete with a basic interface and the most important functionality. But one can almost guarantee that this mock-up will not include some small yet important features that a public, commercial product requires if it's to be used by the masses.

I'm a big fan of Django. But I'm also annoyed at the way it lets us emphasise Pareto's principles. Rapid Web Application Development is the current 'in thing' - using Django or Ruby on Rails, one can get a basic CRUD application online with a minimum of fuss. It works well, it's stable, and it probably has a funky logo and some AJAX magic. There is a good chance it'll have some Google Ads on it to try and cover some hosting costs.

However it'll be missing all the details that make an application truly kick ass. It'll be missing the automatic e-mail notifications that get sent on a monthly basis to subscribers. It'll be missing the extra database fields that make the user experience even better by keeping detailed analysis of user activity to allow the site to be customised to their specific needs. It'll be missing the automated reporting tools that make mundane data truly valuable (both to management types, and to advertisers).

Maybe I'm just impatient, but in this day and age of 'release early, release often' there seems to be too many half-done projects. There are too many almost-there Web-2.0 weblications that, if given that 80% of time, will be something quite new. Earth shattering, even. Something that's ready to make it's developer piles of cash.

It seems there are too many good ideas floating around, but people lose the motivation to get them to that finished and polished stage that makes people say wow. Of course, there are exceptions - just take a look at Tabblo and Flickr to see how a great idea was turned fantastic with the use of some cool technology. Looking at these two sites makes me wonder how many image sharing and social network sites were started but never made it past the beta stage.

Me? I'm working on an online gift registry service for engaged couples. I'm a victim: the site works (and, I must say, works very well), and couples have successfully used it for their bridal registries, but it needs some work around the edges if I want it to be valuable to anybody other than my own small group of friends. There is no reason it cannot be monetized - but people only pay for quality.

Damn that last 20%.

ChangeManipulators on only part of a model

Django has some cool forms management functionality that takes the hard work out of displaying forms and managing the users input. Unfortunately, the ChangeManipulator expects that all fields in a model will be updated by the ChangeManipulator, and cries about any required/not-null fields that you don't display to the user in the form.

This is a huge issue when you want to display a form to a user that only modifies a very small part of a model. For example, you have a 'product' model with a few boolean fields, and you want the user to only alter the 'description' field with a ChangeManipulator. If you were to use the standard ChangeManipulator code (from the above URL) and only displayed the field 'description' to your user, the resulting code would either wipe the values of the boolean fields, or if they're required, complain that they don't have a value.

The fix is simple, and it's been in Django since the new-admin branch late last year. Why it isn't documented, I don't know. It's somewhat explained in Django Ticket # 420.

When you call your ChangeManipulator, there is a second parameter you can use to define which fields will and won't be edited, named 'follow'. It can be used to control multi-level relationships (see the ticket for details) but in this case we want it to ignore a few non-user-editable fields:

follow = {'paid': False, 'package': False, 'reminder_list': False, 'reminder_guests': False, 'security_basic': False}
manipulator = Account.ChangeManipulator(user, follow)

What this will do is ignore the paid, package, reminder_list, reminder_guests, and security_basic fields - and the user won't be prompted with an error message for not entering required fields.

What I'd like to see is a way to exclude all fields (without writing a loop to manually make them all 'False'), and then only include a small subset of fields. In my example, I have three separate screens to edit different sections of the same model - for technical reasons the information is on one model, but for business reasons it's spread over a few screens Using {'*': False, 'editthis_field': True}_ would kick ass.

Sending E-Mails via Templates

So you've got an application written in Django that needs to send large bodies of e-mail, but you don't want the e-mail message itself to be in your Python code. Fair enough, I'd say - you should be separating form from function, and in this case, the e-mail output is still what I'd classify as 'form'.

One way to tackle this situation is to create a template for your e-mail body, process that template to fill in the gaps (eg Username, URL's, etc) and shoot it off via Django's e-mail functions instead of rendering it in a web browser as you'd normally do with templates.

First things first - create your template. In this case I'm writing an e-mail to a user thanking them for registering on a website. I've put the template in my templates directory, called email.txt in the registration subdirectory.

Dear {{ name }},

Thank you for signing up with {{ product_name }}.

Your new username is {{ username }}, and you can login at {{ login_url }}. Once logged in, you'll be able to access more features on our website..

We hope that {{ product_name }} is of good use to you. If you have any feedback, please respond to this e-mail or submit it to us via our website.

Regards,

{{ product_name }} Administration
{{ product_url }}

Pretty simple huh? We're using the same syntax used by standard Django templates, where variable names are enclosed in double-curly-braces.

Next, in your code where you want to send the e-mail, load and compile the template and template context:

from django.template import loader, Context

t = loader.get_template('registration/email.txt')
c = Context({
    'name': new_data['first_name'],
    'product_name': 'Your Product Name',
    'product_url': 'http://www.yourproject.com/',
    'login_url': 'http://www.yourproject.com/login/',
    'username': new_data['username'],
})

What this will do is load your e-mail template into memory, and save it in an object t. Next, it creates a template context, c, with 5 variables in it. These variables share the names in the template above - and in this example, three of them (product name and the two URL's) are hard-coded, and the other two come from a dictionary called new_data.

Lastly, we need to join the context and the template together, and send it off in an e-mail:

from django.core.mail import send_mail

send_mail('Welcome to My Project', t.render(c), 'from@address.com', [new_data['email']], fail_silently=False)

Most of this is pretty straightforward - the e-mail subject will be Welcome to My Project, and you've defined a from e-mail address and recipient (again from the newdata_ dictionary). The funky business is the rendering of the template. The t.render(c) portion simply tells Django, that for template t (defined above), it should use the context c (also defined above), substitute the variables, and return the output. This output becomes the body of the message that the user receives:

Dear Ross,

Thank you for signing up with Your Product Name.

Your new username is rossp, and you can login at http://www.yourproject.com/login/. Once logged in, you'll be able to access more features on our website..

We hope that Your Product Name is of good use to you. If you have any feedback, please respond to this e-mail or submit it to us via our website.

Regards,

Your Product Name Administration
http://www.yourproject.com/

Good luck!

Using Django's TemplateTags

I've had a number of e-mails about how I include the listing of blog tags and archives by month on the side of my website from people who have obviously built up a blog and now want further integration with their website.

Well, it's ultra-simple thanks to a nifty Django feature called template tags. The concept behind template tags is simple - a quick snippet of code in your page templates calls some python code behind the scenes, which does some stuff, and returns either raw HTML code (yuck) or sets new variables in your template context, allowing you to manipulate and display them as you please (cool!)

To get started, in the application directory for your blog, create a directory named templatetags and place an empty file in it, named _init.py_. Now, create a file - in this case we'll call it blogmonths.py_. We need to do a few things in this file:

  1. Import the relevant models so we can access the data
  2. Create and register new template tag
  3. Write the function(s) for that tag so they add data to the template's context.

The contents of this file need to be:

from yourproject.blog.models import Tag,Post
from django.template import Library,Node

register = Library()

def build_month_list(parser, token):
    """
    {% get_month_list %}
    """
    return MonthMenuObject()

class MonthMenuObject(Node):
    def render(self, context):
        context['blog_months'] = Post.objects.dates("date", "month")
        return ''

register.tag('get_month_list', build_month_list)

The important bits of this code are the registration of the tag, and the MonthMenuObject updating context[] and then returning nothing at all.

The context that is set is another Django shortcut - it'll return a date object for each unique month that has a post in it, based on the 'date' column in our Post model. Neat.

Next, create a new file named, for example, blogtags.py_ and paste the following into it:

from yourproject.blog.models import Tag,Post
from django.template import Library,Node

register = Library()

def build_tag_list(parser, token):
    """
    {% get_tag_list %}
    """
    return TagMenuObject()

class TagMenuObject(Node):
    def render(self, context):
        output = ['']

        for blogtag in Tag.objects.all():
            number = blogtag.post_set.count()
            if number >= 1:
                output.append(blogtag)

        context['blog_tags'] = output
        return ''

register.tag('get_tag_list', build_tag_list)

As you can see, this is very similar to the months list - except it checks to see if each tag has any posts, and if it does it adds that tag to a list. That list is then set in the template context for later use.

And that's the hard bit!

Back in your base.html template, choose where you want your list of months to be, and drop in the following code:

<ul>{% load blog_months %}{% get_month_list %}
{% for month in blog_months %}<li><a href="/blog/{{ month|date:"Y/M"|lower }}/" title="{{ month|date:"M Y" }}">{{ month|date:"M Y" }}</a></li>
{% endfor %}</ul>

That'll load up your .py file, execute the tag registered as getmonth_list_ (which, according to the code above, will set a context variable called blog_months), then run through each month in the list and add a link to your page in an unordered list. We use the standard django template filter named date to format the date for valid use in the links and in the anchor text.

Next, choose where you want your tag list and add this (strikingly similar) block of code to the template:

{% load blog_tags %}{% get_tag_list %}
>ul<
{% for tag in blog_tags %}{% if tag.slug %}>a class="link" href="/tag/{{ tag.slug }}/" title="{{ tag.description|escape }}"<{{ tag.title }}>/a<>/li<
{% endif %}{% endfor %}>/ul<

Just like the month block, that runs your new template tag then turns the Python list into a nicely formatted list of tags.

And that's it - you've now got your blog integrated into your website in areas where the blog isn't actually being loaded via generic views - those links are available anywhere in your site!

A Django Blog: Redux

At the start of May, the Django magic-removal branch was merged with 'trunk' - the mainstream Django code base. While this branch isn't yet available for download by the lay user, it's there in svn ready to be checked out and used, and it seems quite stable - the djangoproject.com website was recently moved across to that very codebase.

All of the documentation on djangoproject.com refers to the MR codebase, and because trunk had been updated it meant if I wanted any updates to Django (I run off of the SVN codebase, not the mainstream download) I'd have to go MR. With the 'RemovingTheMagic' wiki page open in one brower screen, I got to making the required changes in another screen. What you're seeing here is the end result of that.

As such, most of the code has changed. Instead of writing a blog entry with a few hundred lines of python code, I've put it all in an archive you can download from my site.

This code is released under a Creative Commons License, as linked with the button on the right side of this blog. Please respect this license - it's very generous. I just want this code to be used as a learning tool, not a money making scheme for somebody :)

I don't intend for this code to be just dumped on a website and used as-is - I expect you to be using it as a learning tool, as a base for bigger and better things. As it is in the archive, it's very basic. The templates aren't complete, they require you to already have a 'base.html' template. This archive also expects you can create a new app in Django and copy these files to the right places.

Good luck, and enjoy your Django experience! I will be making further posts in the future regarding template tags, RSS feeds and other such niceties, which will be based off this code archive.

That's it for now - go and download django-blog-rossp.org.tar.gz.

XSS Vulnerability

A week or so ago I received an e-mail from a nice new zealander, Simon Greenhill, alerting me to a cross-site scripting vulnerability in the comments portion of my blog. The vulnerability will actually probably be of concern to anybody using the comments module from Django - I haven't dug far enough into it as yet to confirm exactly what's at risk.

If you read on, you can see the contents of the e-mail he sent through to me. Basically, the 'name' field from the comments needs to be escaped at the time when the comment preview is displayed. I patched my code some time ago, but now I've got the chance I'm making it public to help everybody else out.

His e-mail said this:

However - I've noticed one problem - your comment preview form is at risk of XSS. If I enter javascript into the name field ( e.g. <script>alert( 'hi');</script> ), it'll be executed. Easily fixed escaping - change this line:

<p>Posted by <strong>{{ comment.person_name }}</strong></p>

to:

<p>Posted by <strong>{{ comment.person_name|escape|urlizetrunc:"40"|linebreaks }}</strong></p>

If this isn't caught at the preview form stage, then it's going to affect your comments listing too since, again, comment.person_name isn't escaped.

The things in comment_form don't seem to be susceptible to this, so I think Django's manipulator's are taking care of the form fields.

So, if you've copied my comments code off this site, it's probably worth making that change for now, until at least Django automatically handles this (to be honest, I thought django.contrib.comments would automatically do this - but I'm sure there's a reason against it) or I can find a 'better' way around it.

More...

Want to see more? Check out the yearly archives below.