WhisperGifts: The Tech That Let Me Launch

The WhisperGifts re-launch recently went very well! I promised a more technical follow-up with some details about what's new and what's changed, so if you want to know more about what makes WhisperGifts tick then you'll want to read on. Hint: It's a dash of Django, a pinch of Python, and a slathering of open-source software all around.

The primary technology behind WhisperGifts is Django, the "web framework for perfectionists with deadlines". My own deadline for this project was rather, ahem, flexible - the new design was a work in progress for 2 years (on-and-off, mostly off due to the birth of our baby) and the back-end re-write happened over a few months early in 2012.

Django allows us to iterate quickly. I find the language natural to use and the documentation is epic. There's a number of things that no framework does out of the box; I've tried to rely on open-source to fill as many gaps as possible rather than re-writing things from scratch like I did with the original WhisperGifts site - this is mostly because the open-source ecosystem around Django is now so much larger than it used to be.

As an example, originally I rolled my own authentication as the user management modules in early Django releases were rather inflexible. Building your own authentication is never a good idea, so I've migrated to using built-in Django logic. Similar changes are found throughout my codebase.

What I Use

Django, obviously. I use most of what comes with Django: The ORM and URL dispatcher, the included Admin, User and Cache apps, and more. Some might be interested to note that I don't use class-based views, simply because I don't see a need to change at this point.

Caching is done using Memcached and nested cache tags, as I've blogged about previously. I also use Django's site-wide caching middleware for anonymous users, which reduces load time for the marketing/static portions of the site.

Images are processed via the sorl-thumbnail package. I can generate thumbnails in any size on the fly. All of my images are stored locally - due to my current volume, the overhead of setting up a more formal CDN or even just using S3 isn't worthwhile.

Customer payments (for upgraded packages) are handled by PayPal. To interface with their IPN and to simplify the user-facing workflow as much as possible, I use django-paypal.

To track in-app metrics (such as number of signups, number of upgrades, number of new items) I use django-app-metrics and get a simple daily e-mail. I'm also testing out Mixpanel which although it isn't free lets me get much more detailed statistics for the same sorts of metrics. django-app-metrics even has a backend to automatically push data through to Mixpanel, so I might use that later on.

All e-mails are sent asynchronously (using django-celery) so they don't tie up front-end web serving. I deliver my e-mails via Postmark with the useful django-postmark library. All my outgoing e-mails include both HTML and plain-text components; I also embed a header image. In the geek world this is heavily frowned upon, but remember who my audience are: couples getting married and their wedding guests. Postmark makes these e-mails simple.

The front-end web server to all my websites is always nginx. It is small, easy to configure, does a wicked job of serving up any static files (both my own site static files and those customer images that have been uploaded) and integrates well with Django. To run Django for nginx I always use gunicorn managed by superisord.

My site-specific CSS and JavaScript files are hand-crafted during development then at runtime combined together effortlessly and minimised as much as possible using django-compressor.

To make sure that any gremlins are caught and dealt with, django-sentry catches any exceptions in my code and presents them in an interface that is incredibly useful: You can see which exceptions occur most often, what conditions trip them, and more.

In a similar vein I use django-timelog and occasionally review how long my views are taking to render in a live environment, django-debug-toolbar gives me similar data during development.

Bringing it all together

For much of the above all I need to do is pip install django-compressor and add the relevant code to my settings.py and templates. Very little of what I've mentioned above has changed the way I develop or deploy; they simply make life easier. The fact that I can pick up these bits of software (most of which weren't available 3-4 years ago) and use them off-the-shelf with some very minimal setup just makes me love Django development even more.

I therefore owe a big "thank you" to the Django community.

Previously I've manually written code to do many of the things I've mentioned above (and bad code, at that, given it's status as "helper code" rather than the main part of my projects). I owe a few people a beer or three.

WhisperGifts Re-Launch

Way back in 2007, my wife Lauren and I got married and went looking for a good bridal gift registry service. We didn't find one, so we built our own - WhisperGifts was born. It's now 5 years later, and today we are re-launching with a completely new look & feel and a 100% new code base under the hood.

I'm new here - what is WhisperGifts?

WhisperGifts lets couples who are getting married put their gift registry online - i.e. make available to their guests a list of the wedding gifts they'd like to receive.

In many cultures this is done by utilising the services of big chain department stores. This has two major downsides: the items are typically more expensive, and the range is moderate at best. Why not have a gift registry that lets you get gifts from anywhere?

So we built WhisperGifts - an online service that lets you list items from anywhere you like. In fact, you don't even list where the item is from - you simply say "We'd love a toaster that cooks crumpets." Your guests are still given enough latitude to select which toaster, from where, and at what price. It's win-win!

As a hobby, I can't say that WhisperGifts is too bad. Enough couples make use of it for me to say that it's worthwhile, and it's a great place for me to try out new tech and startup skills. But I've hardly put enough effort into it, which leads us to today: WhisperGifts 3.0.

WhisperGifts 3.0

I re-skinned WhisperGifts a few years back, but I've never been happy with the layout. A visitor to the website isn't really introduced to what WhisperGifts is, and there is no obvious call to action. There's a thousand and one problems that can be pointed out by anybody who has ever worked on a startup.

The old look and feel isn't the only thing that's gone, though. About 90% of the old codebase has been re-written, but not just for the sake of it - until now WhisperGifts has been running on 5-year-old code, a lifetime in web development. Every now and again I make minor updates to keep it compatible with up-to-date versions of Django, but purely for security purposes.

With a new back-end powered by all the goodness of Django 1.4, I'm able to have a website that runs faster, uses niceties like SQL aggregation, has proper form definitions (my old code still has comments referring to forms and newforms as two separate things) and is more maintainable to boot.

In terms of LOC there's about 50% less code to do the same end result.

So we've got a new visual style and a new back-end, but what else?

  • Our old "Premium" package is now "Standard" and a new Premium package has been created. It includes things like printed guest information cards and an unlimited number of items
  • Sharing of registry details via e-mail, Facebook and Twitter
  • Custom domains - so you don't have to have your registry at 'ournames.whispergifts.com'
  • Coming very soon will be custom registry templates

The new website is now live, at www.whispergifts.com. I'll be blogging again shortly with some more technical details of the change, which are probably of more interest to those who read this blog for the Django posts as none of it will be WhisperGifts-specific.

Getting married? E-mail me and I'll see what we can do about a few freebie premium accounts :)

Why you shouldn't buy cheap watches, and investigate odd sleeping patterns in kids.

A fun anecdote, showing why you should investigate issues that occur too regularly to be co-incidence.

December, 2009 - Tasmania, Australia

A few years ago, my wife and I went to Tasmania and walked the stunningly gorgeous Overland Track, a 65 kilometer (5-day) walk through isolated north-west-central Tasmania. If you're into hiking, you need to get yourself to Tasmania.

Wanting to keep track of time, Lauren bought a new watch. It was cheap, plastic, and bright purple - but at least it wasn't an expensive dress watch - perfect for the great outdoors.

Way before dawn one morning at 4am, sleeping in our expensive hiking tent on a camping platform in the middle of the bush (there are tent platforms throughout the park so as to protect the ground - this is a delicate area) the alarm went off. This little purple watch just started beeping, and never gave up.

In a bushwalking-induced tired haze, Lauren tried to disable the alarm. Floundering and quickly becoming frustrated, she commandeered my pocket knife and tried to stab the watch into silence - somehow avoiding tearing a hole in the floor of our nearly-new tent.

A week later, we returned home and packed our hiking gear away in our spare bedroom. Every now and then we take some of the gear out for a weekend, then it goes back into the cupboard. Unused items, such as the purple watch, slowly float to the bottom of our gear bags.

September, 2011 - Melbourne, Australia

In September last year, our little boy was born. We gave him our "spare room", and cleared out some of the junk in the cupboard. Half of our hiking gear got shifted to the garage, but for now half of it stayed at the bottom of the cupboard in the spare room.

Life's different, when you've got kids. Your "room to dump everything you don't need right now" room isn't yours, any more.

(I'm sure you know where this is going)

Saturday, 16th June, 2012 - Melbourne, Australia

Our kid's sick. First he was teething, which seems to hit the immune system pretty hard. So he got a cold, which he promptly gave to Lauren and I. Finally, it morphed into full-blown Tonsillitis, so it's safe to say none of us are getting a very good night's sleep right now.

Because he's been sick, the boy wakes regularly during the night. Poor fella. The really odd thing, though, was that he was waking right on 4am every night. The accuracy of his night waking was uncanny.

As parents, we are trying to teach our son to self-settle. If he wakes during the night, we try to give him 5-10 minutes by himself to re-settle, and much of the time it works. He learns that he doesn't need us by his side to fall asleep, and we get to stay just that little bit warmer.

So when you added all of this up, we had a chain of events something like this:

  • 4:00am The alarm went off at 4am, waking our son. Being in another room, it was too quiet for us to hear.
  • 4:01am We'd stay in bed for 5 minutes to see if he'd re-settle.
  • 4:04am After beeping for a few minutes, the watch would give up and turn it's alarm off.
  • 4:05am After 5 minutes, we'd sometimes have to go and settle him down. In a sleepy haze, we'd notice it was always 4:05am when we went in to see him.

With his illness this weekend, though, Lauren decided to settle him earlier - right away, in fact.

When she got into his bedroom at 4:01am on Sunday morning to help him get back to sleep, she heard it. The beeping. Flashes of pocket knives and cold tent floors came rushing back, and she managed to fish the wash out of it's bag. The "I'm going flat, dammit, gimme a new battery!" state of the watch had re-triggered it's incessant alarm at 4am each day.

Sunday, 17th June 2012 - Melbourne, Australia.

Revenge: we drowned the watch whilst cleaning the dishes.

Domain Names For Sale

I'm cleaning out my servers and have a few domains I want to sell or give away, including opocket.com (and .net, .org); twolines.com.au and thesecretsanta.com.au. E-mail me at ross@rossp.org if you're interested in any of these. (Note: the .com.au domains will require you to have an Australian address & ABN, I believe)

DjangoSites Move Complete

For the past few years I've hosted all of my projects on a single RimuHosting VPS. It's old, running Debian 5.0, and maintenance and upgrades have become headachy. It isn't easy to run Python 2.7 on old Debian versions, and since I only started using virtualenv relatively recently, things were a real mess.

As such, it was time to upgrade the VPS. I made the decision to look around for other hosting options, and eventually settled on Prgmr as the performance-for-dollar ratio seemed much higher to me.

I've taken the opportunity to break things down into multiple smaller VPS units to try and keep maintenance somewhat simpler. I can pull a single site's server down without breaking any other sites, and I can upgrade components individually for a single site.

Prgmr's pricing makes this possible: Their 'base' is very low - you pay $4 for each VPS plus RAM.

Djangosites.org now sits on it's own 512mb Prgmr Debian 6.0 instance. With 128mb taken up with memcached and a small portion to nginx and the operating system, there's plenty of RAM left for my gunicorn worker processes.

So how did I move everything across?

Although it took me 3 days to finish, the actual effort involved was really quite low. I hope these tips help you with server migrations in the future!

  1. In advance, drop the TTL on your DNS records low. I went to 300 seconds, but on the wrong domain. Doh!
  2. Deploy the new server environment. I have a shell script that creates the relevant users, installs all the Debian packages I need, installs system-wide tools like PIP, installs system-wide Python libraries I need, installs VirtualEnv, installs & configures Postgresql & Memcache, etc.
  3. Put your old app in Offline mode. For Djangosites, this means changing the 'accounts' and 'submit' URLs to point to flatpages instead of their views.
  4. Create a virtualenv on your new server
  5. Copy your (now not-being-written-to) database & code to your new server
  6. Configure nginx for the new app, spin up your app instances
  7. From your desktop, change your HOSTS file to let you access the new server directly as if it were live. Test your siste.
  8. Revert your HOSTS file, and update DNS.
  9. Revert your DNS TTL.

Users with stale DNS will continue to see the read-only site on the old server, whilst those with sensible ISPs will see the updated site within a few minutes.

Upgrading at the same time

Because moving code is boring I decided to take the opportunity to upgrade everything to Django 1.4. Djangosites was, until this week, running on 1.3 but it was using a number of deprecated features that wouldn't port to 1.4. It was also using the Postgres 8.3 contrib package for tsearch, which has different syntax to Postgres 8.4's built-in tsearch.

Unsurprisingly, the codebase has dropped due to this simplification. It's also let me clean out the cruft, simplifying things even further.

I also used the move to point DjangoSites at my Sentry instance - previously I just relied on the built-in Django error e-mails. Using Sentry made it easier to catch a few quirky bugs that were leftover from the move & upgrade.


DjangoSites now has it's own VPS environment, standing alone from my other projects (which, incidentally, also have their own VPS environment). This means I can do maintenance on it without impacting any other projects.

I can also manage memory/RAM with less fuss: I just ask Prgmr to increase the ram for one VPS instance, rather than try and juggle the way each application within a single server fights for memory.

At present, even on only a 512mb VPS, DjangoSites has plenty of headroom. With some basic Blitz benchmarking we're good for north of a million hits a day on a commodity hosting account... obviously we're nowhere near that at the moment, but it's good to know we can grow into this new t-shirt.

My monthly hosting costs have also dropped, but overall I've got allocations of more RAM, bandwidth, and disk space.


DjangoSites now has it's own VPS environment, which means my operations time is going to increase. Any updates and patches to non-app code (such as Postgresql updates or OS security fixes) need to be done multiple times, not just once.

I also suck at remembering hostnames, and need to stop instinctively logging into a single host server. I'm sure I'll get over this pretty quickly, though.


I can't recommend Prgmr highly enough. Their prices are great but more importantly the VPS instances I've got are rock solid. They're fast (including the disk IO) and very responsive. My old RimuHosting VPS wasn't too bad, but the IO was certainly nowhere near as good.

DjangoSites Downtime - Server Move

Heads-up: DjangoSites.org will be down, at some point in the next week, while I move it to a new server. Expected downtime is approximately 1 hour to transfer files & database; if your ISP has DNS servers that don't observe TTL then you might not see updates for a while longer.

This is happening as I'm moving DjangoSites to it's own VPS to make it easier to balance available resources between it and my other projects.

Key-based cache expiration with Django

Last week, the team over at 37Signals wrote up an article on their newly implemented Key-based cache expiration system and it hit me: It's such a simple idea with obvious benefits, why hadn't I implemented a similar caching mechanism before? Being a Django user, the Rails code didn't make much sense to me but the concept certainly did - so here's my take on it with a quick Django example.


I've just implemented this caching strategy for WhisperGifts for a re-launch that will go live in the next few weeks. We allow couples to publish an online gift list, then let people select items from that list. Pretty basic stuff, but rendering the gift list can require n+1 queries due to the way that my purchase data is kept. This hasn't been a big issue until now, when I've built new functionality and generally just extended things a bit.

The cache strategy is so simple it's taken longer to write up here than it did to alter my existing codebase.

My basic model is as follows:

  • Registry, the top-level "collection" of items for each wedding.
  • Item, of which there are many for each Registry
  • Buyer, of which there are 0-n for each Item

With that in mind, read on...

Model Setup

The first task to do is make sure your models have a last_modified timestamp. Mine already did, but just in case you need an example:

class Registry(models.Model):
    name = models.CharField(max_length=100)
    last_modified = models.DateTimeField(null=True, editable=False)

def save(self, *args, **kwargs):
    from datetime import datetime
    self.last_modified = datetime.now()
    super(Registry, self).save(*args, **kwargs)

Next, make sure that whenever you save an Item or Buyer the last_modified change cascades upwards to the Registry.

class Item(models.Model):
    registry = models.ForeignKey(Registry)
    name = models.CharField(max_length=100)
    last_modified = models.DateTimeField(null=True, editable=False)

def save(self, *args, **kwargs):
    from datetime import datetime
    self.last_modified = datetime.now()
    registry = self.registry
    registry.last_modified = datetime.now()
    super(Item, self).save(*args, **kwargs)

Note: I'd highly recommend this gets taken care of in an abstract base class. The above is just a (bad) example.

Define your cache keys

At the top of models.py I defined a variable named DATA_SCHEMA_REVISION. For now I increment it whenever I make a change to my schema that should invalidate the entire cache - not that it's a regular occurrence.

On each model, now, define a cache_key property. For example, on my Item model:

def cache_key(self):
    return 'myproject/%s/item-%s-%s' % (DATA_SCHEMA_REVISION, self.id, self.last_modified)

Again, as a good programmer you'll put this in your abstract model, won't you?

Configure Django caching.

This isn't hard. I use memcached. YMMV.

Update your templates.

My existing template was along these lines:

<b>{{ registry.name }}</b>
{% for item in registry.item_set.all %}
    {{ item.name }}
    {{ item.expensive_calculation_and_rendering }}
{% endfor %}

The new, improved, fully cached version:

{% load cache %}
{% cache 9999999 registry registry.cache_key %}
<b>{{ registry.name }}</b>
{% for item in registry.item_set.all %}
    {% cache 9999999 item item.cache_key %}
    {{ item.name }}
    {{ item.expensive_calculation_and_rendering }}
    {% endcache %}
{% endfor %}
{% endcache %}

Notice that we're caching this for a very long time - because the cache key will change whenever our data is changed. If an Item changes, that item's cache key (and the parent Registry cache key) will be changed due to the new timestamp, and they'll be re-rendered. However when we re-render the outer Registry cache fragment, it's primarily made up of still-valid cache components for the other Items.

This is an incredibly simple way to keep your site nice and snappy. There's some fantastic discussion over on the original 37Signals blog post - I strongly recommend you read & understand that post as well as it's comments, then consider my examples above as a bare minimum implementation in Django.

How quick is it?

A brief update, 10 minutes after this post was written. I have done some very rudimentary tests and come up with the following numbers. Note that these are NOT representative, however the difference is very clear. Keep in mind this is running in a development environment with debugging enabled and various other slowdowns not present in production!

Cold cache 17.34 seconds
Warm cache 1.11 seconds

This is the time taken for Django to process the request form start to finish, from the initial HTTP request to returning a complete HTML page. There's plenty of other gains to make but as a starting point this is not too bad at all!


Want to see more? Check out the yearly archives below.