Tag Archive: djangosites.

Djangosites Open Sourced

Back in 2008 I started djangosites.org as a listing of websites powered by Django. Prior to that, we relied on a wiki page to see who was using Django, so an image-based website felt like a big improvement.

Since day one I've promised to release the source code that I use for the site. It's relatively simple, so I never stressed much about making it a high priority - but I continue to be asked and politely berated for not getting it published.

Today that's changed. I think it's too late for me to say I've come good on my promise, but the Djangosites source code is now available on GitHub.

The README has more details, but in short this is a dump of the code currently running the site. I'll continue to use this repository as changes are made to the live site, however I'm not actively working on djangosites at this point in time (other than reviewing & approving submissions)

There's a few pieces of this that might be useful for people new to Django, but otherwise this is really a collection of generic views. The useful bits might be:

Suggestions and pull requests are welcome, but I'm not actively soliciting changes. I should probably clean things up a bit given that this codebase hasn't changed materially since Django 0.96, other than slight refactors to allow upgrades to work - so I'm certainly not yet taking advantage of new functionality that's been made available in recent Django versions. Perhaps now you can see how bad the code is, I'll have more of an incentive to fix it :)

The source code is available right now from GitHub under a mixed licence: the Python code is MIT-licenced, and the rest (HTML etc) is not open source but included in the repository for completeness and as an example.

DjangoSites Move Complete

For the past few years I've hosted all of my projects on a single RimuHosting VPS. It's old, running Debian 5.0, and maintenance and upgrades have become headachy. It isn't easy to run Python 2.7 on old Debian versions, and since I only started using virtualenv relatively recently, things were a real mess.

As such, it was time to upgrade the VPS. I made the decision to look around for other hosting options, and eventually settled on Prgmr as the performance-for-dollar ratio seemed much higher to me.

I've taken the opportunity to break things down into multiple smaller VPS units to try and keep maintenance somewhat simpler. I can pull a single site's server down without breaking any other sites, and I can upgrade components individually for a single site.

Prgmr's pricing makes this possible: Their 'base' is very low - you pay $4 for each VPS plus RAM.

Djangosites.org now sits on it's own 512mb Prgmr Debian 6.0 instance. With 128mb taken up with memcached and a small portion to nginx and the operating system, there's plenty of RAM left for my gunicorn worker processes.

So how did I move everything across?

Although it took me 3 days to finish, the actual effort involved was really quite low. I hope these tips help you with server migrations in the future!

  1. In advance, drop the TTL on your DNS records low. I went to 300 seconds, but on the wrong domain. Doh!
  2. Deploy the new server environment. I have a shell script that creates the relevant users, installs all the Debian packages I need, installs system-wide tools like PIP, installs system-wide Python libraries I need, installs VirtualEnv, installs & configures Postgresql & Memcache, etc.
  3. Put your old app in Offline mode. For Djangosites, this means changing the 'accounts' and 'submit' URLs to point to flatpages instead of their views.
  4. Create a virtualenv on your new server
  5. Copy your (now not-being-written-to) database & code to your new server
  6. Configure nginx for the new app, spin up your app instances
  7. From your desktop, change your HOSTS file to let you access the new server directly as if it were live. Test your siste.
  8. Revert your HOSTS file, and update DNS.
  9. Revert your DNS TTL.

Users with stale DNS will continue to see the read-only site on the old server, whilst those with sensible ISPs will see the updated site within a few minutes.

Upgrading at the same time

Because moving code is boring I decided to take the opportunity to upgrade everything to Django 1.4. Djangosites was, until this week, running on 1.3 but it was using a number of deprecated features that wouldn't port to 1.4. It was also using the Postgres 8.3 contrib package for tsearch, which has different syntax to Postgres 8.4's built-in tsearch.

Unsurprisingly, the codebase has dropped due to this simplification. It's also let me clean out the cruft, simplifying things even further.

I also used the move to point DjangoSites at my Sentry instance - previously I just relied on the built-in Django error e-mails. Using Sentry made it easier to catch a few quirky bugs that were leftover from the move & upgrade.

Positives

DjangoSites now has it's own VPS environment, standing alone from my other projects (which, incidentally, also have their own VPS environment). This means I can do maintenance on it without impacting any other projects.

I can also manage memory/RAM with less fuss: I just ask Prgmr to increase the ram for one VPS instance, rather than try and juggle the way each application within a single server fights for memory.

At present, even on only a 512mb VPS, DjangoSites has plenty of headroom. With some basic Blitz benchmarking we're good for north of a million hits a day on a commodity hosting account... obviously we're nowhere near that at the moment, but it's good to know we can grow into this new t-shirt.

My monthly hosting costs have also dropped, but overall I've got allocations of more RAM, bandwidth, and disk space.

Negatives

DjangoSites now has it's own VPS environment, which means my operations time is going to increase. Any updates and patches to non-app code (such as Postgresql updates or OS security fixes) need to be done multiple times, not just once.

I also suck at remembering hostnames, and need to stop instinctively logging into a single host server. I'm sure I'll get over this pretty quickly, though.

Overall

I can't recommend Prgmr highly enough. Their prices are great but more importantly the VPS instances I've got are rock solid. They're fast (including the disk IO) and very responsive. My old RimuHosting VPS wasn't too bad, but the IO was certainly nowhere near as good.

DjangoSites Downtime - Server Move

Heads-up: DjangoSites.org will be down, at some point in the next week, while I move it to a new server. Expected downtime is approximately 1 hour to transfer files & database; if your ISP has DNS servers that don't observe TTL then you might not see updates for a while longer.

This is happening as I'm moving DjangoSites to it's own VPS to make it easier to balance available resources between it and my other projects.

DjangoSites Deployment Statistics

Every person that submits a site to DjangoSites gets a chance to include details about how they deployed their website: what database they use, what version of Django they use, and so on.

The aggregated statistics are now online for all to see. The deployment details for individual websites are not visible.

You can see the stats in a basic form along with some pretty charts on the DjangoSites website by clicking the Deployment Stats link in the navigation bar.

I'd love to hear any feedback you've got, please let me know your thoughts via e-mail or Twitter

Update Your DjangoSites Screenshots When You Want

You can now update your DjangoSites website screenshot by simply editing your listing and ticking the box that says "Re-do Screenshot".

To edit a website, first log in to DjangoSites using your username or OpenID. Once logged in, click 'My Sites' in the navigation bar, then click on the website you wish to have updated.

From the website detail page, click the 'Edit' link next to your username. Make any changes you wish to make, click the Re-do Screenshot checkbox, and click 'Save'.

It's important to note that when you edit a website, it becomes un-verified so it will not be publicly visible. I manually review all submissions and edits, so it might take me a few days to verify and publish your listing after you edit it.

This is something that has been on the books for a while, and I'd like to thank Richard Barran for prodding me into getting the change done.

While you're editing your listing, you can update your deployment details to help with our Django Deployment Survey. The results will be published shortly for all to see - thank you to those who have responded so far!

The Django Deployment Survey: How Are You Hosting Django Websites?

According to the official Django Documentation, there are around a half-dozen documented and supported ways of deploying Django in production environments. The recommended way these days is with Apache and mod_wsgi, but there are still a whole bunch of decisions that one must make when it comes to making their Django project public.

One of the great things about running DjangoSites is that it has exposed me to a number of fantastic Django-powered websites. Each of these is unique in one way or another, but they are all running Django. How they use it, though, is anybodies guess.

So what are our peers doing? There are proponents for and against each of the different pieces of the Django Deployment puzzle.

  • What operating system should be used? BSD, Linux or Windows?
  • What Database system should be used? Postgres, MySQL, or something else?
  • Which web server software? Apache, nginx, or one of the many python-powered web servers?
  • Which method should I use? mod_python, FastCGI, or something else?
  • Which version of Django? Do I stick to a release, or try and keep up with SVN?

To see what our Django-using peers are doing, I am starting a survey of Django website deployment methods. This will be an ongoing addition to DjangoSites.

The way it works is pretty straightforward. As of tonight, there are a number of new fields on the 'Submit a Site' and 'Edit' screens to let you select the way you have deployed your Django-powered website. I will not be publishing these details on a site-by-site basis, so you can rest easy knowing that I won't tell the world that your website is hosted on Windows with an Oracle database backend.

Once there are a material number of responses to those questions, I will publish the statistics in aggregate with some pretty colourful charts. These graphs will remain public (and dynamic - updating as the data grows) once the volume of submissions provides meaningful data.

What will this give us? An interesting look at the way that the wide public deploy their Django applications, and potentially longer-term trends showing the usage of particular deployment methods as time goes on.

Want to be involved? I'd love to hear your feedback and see as many sites as possible have their deployment details listed.

If your site is already listed at DjangoSites, you can simply log in and click 'My Sites' in the toolbar. Open each site that's listed, and click the 'Edit' link. Alternatively, if you have a whole bunch of websites you want to update, just email me your DjangoSites username and deployment details and I'll do the update for you.

Don't despair if your websites aren't listed at DjangoSites yet - you can simply sign up then submit your websites for free! Remember, the deployment details are optional so you can list your website without being included in the aggregated statistics if you prefer.

Lastly, I would like to thank Joshua Jonah of Symbiosis Marketing for planting the seed for this idea. I'm hoping it'll give an interesting insight into more of the 'behind the scenes' details of Django-powered websites.

Django Site of the Week: Ooh-Ga-Boo-Ga

For the second Django SOTW, I had a chat to Bruce Kroeze about one of his Satchmo projects, Ooh-Ga-Boo-Gah. It's a great match of cool design and technical wizardry.

For the un-initiated, Satchmo is one of Django's 'killer apps', providing an e-commerce platform that enables Django-powered shopping sites to be built with ease. Bruce is on the core development team and was able to provide us with a great insight into the project.

You can read the interview now.

Note: Due to Christmas and New Years, there will be no Django SOTW next week. We'll return on January 3rd, 2009. Have a great holiday season!

Django Site of the Week

Since I started DjangoSites over a year ago, the response has been fantastic. I used to approve a handful of websites a week, recently it's more like a half-dozen a day.

I've decided that a bunch of those websites are just awesome, and I wanted to have a chat with the owners of those sites and share their stories with the Django Community. Most larger or more unique stories have a story behind them, and the experiences of the team building them are vary from project to project.

Upon approaching the owners of a number of cool websites powered by Django, I received great feedback and enthusiastic responses. So far there are a few weeks of interviews ready to be published, with more on the way.

The first website we've featured is Disqus, a hosted comment engine that can be used on any blog or website for free. It's an interesting website, and Daniel Ha (one of the founders) was very helpful in answering my questions and helping to share his experiences with Django.

Each weekend, another interview will be published. You can subscribe to the RSS feed to be automatically alerted to new websites, however I suggest you visit the website to see the full content of the interviews along with screenshots and statistics that have been shared with us.

I would like to take this opportunity to say a big thank-you to everybody who has helped out so far with interviews, as it has made it much easier for me to undertake this project.

Lastly, I'm going to need a continuous supply of new material. If you find a Django-powered website that you really like, please let me know and I will contact the owners of the website. I also encourage you to leave comments on the website including any questions you would like answered in future interviews.

Enjoy!

Django Site of the Week: Suggestions Wanted

Over at DjangoSites there is a steady flow of new websites, with a handful showing up every single day. Something really cool is that a growing number of these are either commercial websites that are using Django to make money, or they are very serious in terms of code-base, development effort and online exposure.

Django is growing up, and is making an impression on the web at large: I've decided to interview the brains behind these websites to find out why they chose Django, how it assisted their development processes, and how it got in their way. Soon I'll be launching Django Site of the Week, and I want your input.

So far I've conducted quick interviews with a number of fantastic websites that are powered by Django. Some of these are commercial and charge their users for access, others are large-scale community projects, and others are a hybrid. I'm in the process of editing them into something useful for the community, with plans to launch Django SOTW within the next few weeks.

Each week, I aim to publish an interview with the creators of a Django-powered website that stands out from the other sites listed at DjangoSites. I'm looking for the real cream of the crop: websites that are more than a blog and a collection of generic views. Some of the websites you can expect to see interviews from in the first few weeks include EveryBlock, SuggestionBox and Disqus. These sites are all unique and have approached a problem in their own way, all of them using Django. The reasons for using Django and the issues they faced are interesting, and some background information should help aspiring entrepreneurs as they work towards releasing their own Django-based websites.

While I continue to prepare the websites, I would like any suggestions for websites to feature. If you would like to nominate a website to be the Django Site of the Week, or if you have a unique story to tell from your involvement in the development of a complex Django powered website, please email me at ross at this domain. I'd love to hear from you.

DjangoSites is 1337

It's kind of nerdy and geeky, but we've just passed the 1337th approved site on DjangoSites.

The lucky submission was Chinet, the website for a set of environmentally friendly disposable tableware.

From glancing at the websites as they're approved, I'm noticing the quality of design of many websites improving compared to the more 'tech-friendly' submissions when the project first began. It's a great sign that Django is reaching further into everyday website development, which can only mean good things.

Here's to another 1000+ sites!

DjangoSites Gets New Shoes

I launched DjangoSites publicly last June, and since then we've had over 1,250 websites submitted to what has become the ultimate directory of Django-powered websites.

The look and feel of the site didn't seem to have the same zing that many of the listed sites had though, so after a quick chat with the ever-creative Martin 'maddiin' Czura he was able to put together a new design that we both think is a great improvement.

The new design is already online at DjangoSites.org, with a few nifty changes as well as the colour scheme such as an improved navigation bar and less cluttered positioning of user account options.

At the same time, the website has been moved to a new VPS with KnownHost. I've been using KnownHost for over a year now and their VPS hosting is fantastic. The support from them has always been helpful, and their prices are very reasonable for what has been a very reliable service.

I updated the TTL for the djangosites.org domain a few weeks back, so by the time this post goes live the new IP address should be accessible to anybody with properly configured DNS servers. I encourage you to do a forced reload (using Ctrl-Reload in FireFox in Windows, I assume it's similar for other browsers) to ensure that the new stylesheet and graphics are loaded.

If you do find yourself getting to the blue design on the old server, you will not be able to submit sites, vote, or leave comments until your ISP realises there has been a change in IP address.

Don't forget, we now have OpenID support so you don't need a username and password to sign up. Since my announcement that OpenID was available a fortnight ago, there have been 44 signups of which 18 have used OpenID exclusively (not including users who added an OpenID to their existing account). It's fantastic to see this type of momentum behind OpenID - it would be interesting to see similar statistics from other websites.

As always, any problems or suggestions can be sent to me at djangosites at djangosites.org. Enjoy!

DjangoSites Gets OpenID Support

OpenID is, in my opinion, critical in the success of an open web. It allows an individual to access websites without providing that website with a password, and it provides a single identity across non-homogeneous websites.

Generally speaking, it also makes the signup process for a website much simpler. Rather than the traditional method of finding a username that isn't yet in use, entering a password, verifying your e-mail address, then selling your firstborn, with a system such as OpenID you simply enter your OpenID Identifier (typically, but not necessarily the web address of your weblog) and click 'Signup'. You then verify your username and password with your OpenID Provider (the only username & password you should have to remember) who returns you to the original website with a token saying "Yep, this really is the guy who says he's rossp.org."

There are plenty of descriptions of OpenID works, so I won't harp on about it too long. Lets just say that I think it's an important development in todays web.

Django got it's first dose of the OpenID 'syrup' from Simon Willison, who released his efforts as django_openidconsumer. This application lets you use OpenID on your website, however it doesn't provide any integration (yet) with the existing Django Authentication framework.

Soon after Simon's release, a number of people provided their own ways of integrating OpenID with Django's auth, including a messy codebase that I wrote. I personally didn't think any of these were ready for the prime-time, so until now I hadn't implemented OpenID on any public Django facing websites.

That changed recently when I came across django_authopenid, written by Benoit Chesneau. His code is much more complete than mine, and with a few minor changes (most of which I've submitted back to the project for others to use) I've been able to very easily add OpenID support to DjangoSites.

As of today, you can continue to use your existing username & password, or you can use OpenID. To use OpenID, simply log in using your OpenID and you'll be given the option to attach your OpenID to an existing account (if you've already signed up with a username & password) or create a new account tied directly to your OpenID (no password required!).

Once you're logged in, there are also a few new user features including easier access to your previously submitted sites, the ability to change your password, and the ability to delete your account entirely if you wish.

OpenID is starting to make more of an impact on various websites. Simon Willisons' Django People website already has OpenID support, and I hope others in the Django community follow his excellent lead.

Overdue Catchup

I've had a very busy few months in every way conceivable - everything from my Django projects, to my day job, to life as a whole has been running in fast-forward. Here's a quick summary of the

DjangoSites is coming along very well, with 1040 sites listed as of this evening. The quantity and quality is ever-increasing, and more and more sites are being claimed. There are still well over 300 unclaimed sites - is yours listed there? If so, drop an email with your DjangoSites username to djangosites@djangosites.org.

After a server move late last year my Django OpenID project went offline for a little while. After a handful of requests from the blogosphere I've put it back online - see my original blog posting on the topic for more details, although I'm guessing that Simon Willison has something up his sleeve that'll trump my hack-job soon enougy.

Over my Christmas holidays I launched Jutda, the 'corporate' face for my upcoming web projects built with Django. The word Jutda is from the Wagiman language, a dialect spoken by an ever-shrinking Aboriginal tribe in the Northern Territory of Australia. It means show the way, which is something I hope to do with my projects. This is by no means my day job, rather a single name with which to pull together a number of after-hours projects. Hopefully the name starts to mean something within the Django community after a little while :)

The first project to be released by Jutda is WhisperGifts, a service to allow you to publish your wedding gift registry online with minimum fuss. I used it for my wedding almost a year ago with no problems, and it's been used by others before and since with great praise from happily married couples and their guests alike. Of course, it's all built with Django, with a whole host of neat features. When I have a spare evening I'll write up a few more details, but in the mean time check it out and let me know what you think.

Last but not least, I thought I'd bring attention to a simple Django application I wrote a while ago but never 'released' per-se. Django-forum is a simple Forum application for Django, allowing you to leverage your existing templates and user accounts to add discussion-forum capability to your existing project. At least a few people are using it, with a few patches coming from the community to add new functionality. I don't currently have any public sites running it, but keep your eyes peeled.

That's all of my news for now - I've got plenty more to share, however just a little more time is needed to give a few projects some more polish before I go public. Let's just hope my next update isn't another three-months away!

DjangoPoweredSites Grows Up

After some e-mail discussions with Jacob Kaplan-Moss from the Django team, I've moved all of the sites listed on the old DjangoPoweredSites wiki page to Djangosites.org.

It took a fair bit of time to weed out some not-working pages and expired domains (which have, of course, been purchased by spammers and link farmers), and a small investment in extra WebThumb credits to allow me to take screenshots of a fresh 500+ websites, but I got there in the end.

We'll now be closing the DjangoPoweredSites wiki page for good, as it has become redundant and is not the best way to peruse 800+ websites.

What we're left with is a fantastic directory of websites powered by Django. I'll be putting together a 'Featured' list shortly, but in the meantime I need a little bit of help from the community.

All sites scraped from the wiki page are currently unowned. I encourage you to check out the list of unclaimed websites, and e-mail me to let me know which sites are yours so you can receive due credit. There are some great websites listed there, so it'd be fantastic to be able to show off who built them.

Thanks to Jacob and the rest of the Django team for getting behind this initiative.

Django: A Diverse Community

Scott Barnham, one of the guys behind the recently-launched Django Gigs website, has posted some statistics from visitors to the Gigs website over the past few days. I've put together some similar stats for the Django Sites website, which has been online for a few months now.

Note: All figures are from Google Analytics, and percentages are rounded to make life easier.

Summary

Operating Systems

Windows 60%

Linux 21%

Macintosh 18%

Others 1%

The operating system split is very much skewed towards Windows, almost all of the Windows users on Windows XP. It's nice to see higher-than-average (compared to my other sites) usage of Linux & Macintosh.

Browsers

Firefox 66%

Internet Explorer 16%

Safari 7%

Opera 5%

Others 6%

No surprises here. Firefox figures are very similar to the Gigs site, with Safari & IE being a bit different. I'm guessing the higher usage of IE shows the target audience of DjangoSites - people who are looking for what Django is capable of, perhaps newer-comers to the Django world rather than long-term stable mates who may be more likely to use Firefox or Safari on their shiny Macbook Pro.

Geographical Areas

North America 30%

Western Europe 18%

Eastern Europe 9%

Southern Europe 9%

Northern Europe 9%

South America 6%

Eastern Asia 4%

Australia & New Zealand 3%

Elsewhere 12%

Again, this isn't really too surprising. There is a very long-tail (and I'm not overly sure Google Analytics needs to break down Europe & Asia into so many sub-continents), easily visible with "Elsewhere" covering 12% of the visits.

Stuff I Found Interesting

  • Bolivian visitors viewed 20 pages/visit, followed by 18/visit for Oman. Everyone else was miles behind this.
  • 52% of traffic is from referrals, most of it from the Django website, an article in Smashing Magazine, and Wikipedia.
  • 30% of traffic is search-engine generated, typically from searches for 'django sites' - and somehow, including a few searches for 'Auction'.

Hopefully these numbers are interesting to people looking to target their offerings to the Django audience. Do they match your sites?

DjangoSites: We Want YOU!

I've just approved another batch of sites bringing the total to 260. I've also had to delete a few sites that were submitted due to a number of reasons: A handful were submitted that are obviously not Django (providing your link as /index.php is a give away) and a few have been submitted that are inaccessible URL's.

If you've got a website built with Django, we'd love for you to submit it. There are about 600-700 sites listed on the wiki page, if we can get most of those moved over to DjangoSites.org it'd be fantastic. It's a great way to show off what's out there in the wild that's powered by Django, and a great way to get feedback on your sites.

For those learning Django, it's also a great way to see how others have built their sites. Our listing of sites with publicly available source is growing every week, and is a great way to view how different problems have been tackled.

In the meantime, maddiin and I are working on a repository for Django-powered applications. There are plenty of projects out there (especially on Google Code), however having listed together in one place can only be a good thing. At a glance, you'll be able to see what version of Django and Python you need for a given application, and you can set your user preferences to only show applications that run on your version of Django so you aren't constantly looking at potentially incompatible applications. Keep your eyes here for more upcoming information about this.

Djangosites Updates

Last week I launched djangosites.org, a new website to show off other websites built with Django. Uptake has been great (as I write this there are 125 websites listed) and feedback has been even better.

Firstly there are more RSS feeds. You can view RSS feeds for latest entries, particular tags (eg business), or individual authors. The 'RSS' button at the top-right of most pages links to an RSS feed of your current view, more will be coming soon.

Next, listings with source-code available are now more easily identifiable from the listings by a new icon: (a good example of this is the blog tag listing which has a few listings with source-code available)

We've also done a fair few minor cosmetic changes, eg signup emails now come from a sane email address, and the signup/login forms match the rest of the site

Any other suggestions are more than welcome, in the meantime maddiin and I are working on our next Django community project! Stay tuned for more details.

Introducing Djangosites.org

Until now, if you wanted to see a few sites that were built with Django you had to wade through a list of a few hundred links on a wiki page. The links were somewhat organised into categories, but no matter who you ask that's a boring job.

Djangosites.org has been built as a showcase of what's out there in Django Land.

We hope it lets you easily see what's capable with the Django framework. Feel free to add your sites, post comments on other sites, and vote for your favorites. What I'd really love is if some of the Django heavyweights could post some sites up - it would kick ass if this can eventually replace the DjangoPoweredsites wiki page.

Thanks to maddiin for the site design, and to the django-users mailing list for feedback and suggestions. If you have any feedback please don't hesitate to email it to me at ross at this domain.

More...

Want to see more? Check out the yearly archives below.