Posts Tagged Django

First version of RankBuzzard.com

April 14, 2009 at 4:02 pm · Filed under rankbuzzard.com ·Tagged , , , , ,

I spent the holiday weekend implementing a website using the Django web framework.  In just a few days, I managed to put together a fully functional website that displays Google Hot Trends data and collects user comments.  The general point might be described as “Why are these searches popular?”  You can see the results at RankBuzzard.com.

I last built a public website roughly 4 years ago, when I wrote imwatching.net (which is no longer up).  At a broad level, the two websites are similar.  Both collect time series data, put it in a database, and present it via HTML.  I built imwatching.net using a collection of perl scripts that used the CGI module.  I had no ORM.  I had to build my own user management logic.  I had no templating system.  As a result, it took me at least 3 or 4 times as long to develop imwatching.net.  The end result also wasn’t nearly as tidy and well structured as my implementation of RankBuzzard.com.

What’s more, I hosted imwatching.net on a dedicated server that I rented from serverbeach.com for roughly $100/month.  Although I was happy with the quality of the server, the $100/month cost ultimately caused me to close the site.  Now I’m paying $20/month for a virtual server at linode.com.  Although it’s not a direct comparison, I’m been very happy so far with the service at linode.com.

Hopefully some people will find RankBuzzard interesting, informative, and fun.  I have several ideas for how to improve it, and I plan on adding to it when I find the free time over the next few weeks.

Leave a Comment

Manage your Django Transactions

January 28, 2009 at 12:41 am · Filed under Python ·Tagged , ,

I’m building a Django application at work that performs several thousand inserts.  I was frustrated when it was taking several minutes to populate the database.  For reference, I’m running on a Windows XP box with a 2GHz Xeon, 4GB of RAM, and I’m using SQLite as the database.  In one straightforward piece of the program, it was taking over 3 minutes to insert 5000 rows.

Since I’m new to Django, I wasn’t immediately aware that, by default, every call to an ORM object’s save() results in a separate transaction.  When you’re doing thousands of inserts, the overhead is unbearable.  So after reading this, this, and this, I realized that the @transaction.commit_on_success decorator would allow me to lump a collection of saves into a single transaction.

from django.db import transaction

@transaction.commit_on_success
def myfunction(request):
    for i in range(5000):
        myobj = MyObject(mynumber=i)
        myobj.save()

After I made this change, the 5000 inserts took roughly 15 seconds instead of over 3 minutes.

Comments (2)

Follow

Get every new post delivered to your Inbox.