Archive for January, 2009

Open Source CDS Pricing

January 29, 2009 at 11:40 pm · Filed under Business ·Tagged , ,

I just read an ISDA press release saying that the J.P. Morgan CDS Analytics Engine will soon be open source.  I hope it’s true because I’d love to feast my eyes on that code base.

For the uninitiated, CDS stands for Credit Default Swap.  Put simply, a CDS is an insurance contract between two parties that references some financial contract, such as a bond.  The party selling insurance receives payments from the buyer every quarter.  If the referenced entity (bond) defaults, the quarterly payments immediately stop, and the insurance seller pays out a large lump sum to cover any losses.

CDS contracts have a value that fluctuates with variables such as time, interest rates, and the perceived risk of default on the referenced entity.  As a result, there is a large, liquid market for these derivative contracts.  A single contract is typically written to insure a notional of $10, $20, $50, or $100 million.  Despite dealing with sums that large (or larger), there is no reference, open source pricing mechanism.

Said another way, if a bank or hedge fund wants to value one of its CDS positions, there’s no reference, open source pricing algorithm.  Instead, each desk might use an expensive, commercial library, or the CDS pricing screen on their (expensive) Bloomberg Terminal, or their own proprietary pricing algorithm.  The standard pricing model within the Bloomberg Terminal is their “J.P. Morgan model,” the details of which are only described at a high level in a one page document.  You can’t access the code.

At a previous job I wrote a proprietary CDS pricing algorithm.  I was shocked at how subtle variations in the algorithm produce signficantly different results.  Furthermore, it’s impossible to tie out your results exactly with a standard such as Bloomberg’s pricer.  And I’m wasn’t alone.  I spoke to quants at other banks and to vendors who sell their own pricers, and none of them were able to exactly match Bloomberg’s pricer!  I had a conspiracy theory that Bloomberg made their pricing output cryptographically secure so that traders would be forced to pay for their CDS pricer.

So if this J.P. Morgan code has any relation to the elusive code within the Bloomberg CDS screen, I can’t wait to see it.

Leave a Comment

Manage your Django Transactions

January 28, 2009 at 12:41 am · Filed under Python ·Tagged , ,

I’m building a Django application at work that performs several thousand inserts.  I was frustrated when it was taking several minutes to populate the database.  For reference, I’m running on a Windows XP box with a 2GHz Xeon, 4GB of RAM, and I’m using SQLite as the database.  In one straightforward piece of the program, it was taking over 3 minutes to insert 5000 rows.

Since I’m new to Django, I wasn’t immediately aware that, by default, every call to an ORM object’s save() results in a separate transaction.  When you’re doing thousands of inserts, the overhead is unbearable.  So after reading this, this, and this, I realized that the @transaction.commit_on_success decorator would allow me to lump a collection of saves into a single transaction.

from django.db import transaction

@transaction.commit_on_success
def myfunction(request):
    for i in range(5000):
        myobj = MyObject(mynumber=i)
        myobj.save()

After I made this change, the 5000 inserts took roughly 15 seconds instead of over 3 minutes.

Comments (2)

You Can’t Cheat an Honest Man

January 27, 2009 at 4:57 pm · Filed under Business ·Tagged ,

There was a long investigative reporting piece on Bloomberg today about Bernie Madoff’s enablers.  The article makes the point that many of Bernie’s investors and feeders suspected Bernie was a cheat, but they didn’t mind as long as they benefited.  For example, it quotes a fund of fund’s executive as responding to an accusation that Madoff was front-running his brokerage clients by saying, “Yeah, so what?  That’s his edge.”

The article mentions a book, by James Walsh, titled You Can’t Cheat an Honest Man.  I like the title, which he borrowed from a 1939 W.C. Fields movie.  Although I feel badly for the people whom Madoff victimized, I think that the phrase rings true.  For years people had been claiming that Madoff’s returns weren’t feasible, but investors piled on anyway.  Some feeders, such as Fairfield Greenwich, “put 53 percent of its assets into Madoff’s hands and charged clients 20 percent of profits; it added a 1 percent management fee in October 2004.  For the $7.3 billion funneled to Madoff, they reaped $102 million per year, assuming Madoff returned a phantom 10%.

Not all people went along for the ride though.  The article reports that Oswald Gruebel, who was head of private banking at Credit Suisse, suspected something was wrong and “urged customers to withdraw from Madoff’s funds.”  Apparently only about half of the money was taken out.

Incidentally, a google search for the phrase, “you can’t cheat an honest man” turned up this prescient editorial from November, 2005 by Christopher Laird.  He wrote:

The US right now is in the same economic boat. A 5 year housing bubble has caused real estate to rise in most cites by over 100%, and because real estate is easy to leverage, people using historically low interest rates have basically succeeded in mortgaging their houses well past levels that would be sustainable.

Now people have been enticed into an endless cycle of buying on credit cards and sold the lie of using a home mortgage to pay those off, only to begin the cycle again. And now we have these new bankruptcy laws that are waiting to lower the boom on the many people who were enticed and unwisely accumulated lots of debts, having been seduced by a chimera of rising housing values that are only going to fall off a cliff once the people realize the jig is up. 90% of them will lose their shirts… sadly….

Leave a Comment

Hack for functools.partial and multiprocessing

January 23, 2009 at 5:04 pm · Filed under Python ·Tagged ,

Today I wrote some python code that mapped a function onto a sequence of financial time series data.  For the first version, I used itertools.imap to apply a partial function created with functools.partial to the sequence.  I needed a partial function with curried arguments because a couple of the inputs were created and fixed before the call to itertools.imap.

The program took 20 minutes to run because the dataset is rather large.  Since it’s an embarassingly parallel problem, I decided to give the multiprocessing package a whirl.  I’ve used multiprocessing’s parallel map functions before, so I naively created a worker Pool with 4 proceses and changed itertools.imap to pool.imap.  UnfortunatelyI was smacked with this error: “TypeError: type ‘partial’ takes at least one argument.”

It seems that the functools.partial object isn’t pickeled across the multiprocessing workers.  I worked around this by using itertools.izip, combined with itertools.repeat.  The itertools.repeat function mimics currying by repeating the fixed argument.  I’ve pasted a simple example of this below, and I’ve also put it here.  If anybody has a better way of doing this, please let me know.

import functools
import itertools
import multiprocessing

def uncurryDummyMultiply(t):
    return dummyMultiply(t[0], t[1])

def dummyMultiply(a, b):
    return a * b

def serialMap():
    mathFun = functools.partial(dummyMultiply, 10)
    total = 0
    for x in itertools.imap(mathFun, xrange(1000)):
        total += x
    return total

def parallelMap_Error():
    """Generates an Error!
    TypeError: type 'partial' takes at least one argument
    """
    mathFun = functools.partial(dummyMultiply, 10)
    pool    = multiprocessing.Pool(processes=4)
    total = 0
    for x in pool.imap(mathFun, xrange(1000)):
        total += x
    return total

def parallelMap_NoError():
    """Parallel version of serialMap that doesn't generate a TypeError
    """
    pool    = multiprocessing.Pool(processes=4)
    total = 0
    for x in pool.imap(uncurryDummyMultiply,
                       itertools.izip(itertools.repeat(10),
                                      xrange(1000)),
                       chunksize=50):
        total += x
    return total

if __name__ == "__main__":
    print "serialMap result:   %d" % serialMap()
    print "parallelMap result: %d" % parallelMap_NoError()

Leave a Comment

Blurry image on nytimes.com

January 8, 2009 at 9:36 pm · Filed under Random Thoughts ·Tagged

For a while now, nytimes.com has linked to their headline opinion piece with a blurry image.  For whatever reason, it seems their designers simply magnify a screen capture of a their standard small font.  The grainy result is unsettling because it’s sandwiched between normal, smooth images.  I wonder why they do this, and I wonder if this bothers anybody else.

blurry image on nytimes.com

Here is a screen capture taken within Firefox on my MacBook Pro.

Leave a Comment

Evernote rocks, but it could improve

January 7, 2009 at 2:30 pm · Filed under Business ·Tagged ,

For that past few weeks I’ve been using Evernote to keep a running journal both at work and at home.  As a former user of Microsoft OneNote, I’m delighted to see how well this relative newcomer holds up in the domain of note taking applications.  It impressed me so much that I decided to toss them $45 for a year of premium user status.  That said, Evernote has room to improve.

Note taking programs help me be more efficient at juggling multiple projects.  I work at a trading desk where I develop software, do research with coworkers, and execute trades in the market.  Organization doesn’t come to me naturally, so I use note taking software to store a running journal of thoughts, to-dos, issues, and solutions.  I like the ability to insert primary sources such as emails, webpages, screen captures, documents, or pictures into the journal.  Then I can use tags, timestamps or text search to retrieve it all later.

On the whole, Microsoft OneNote is a great piece of software.  My primary complaint is that it’s a Windows-only application.  It also isn’t free.  My desktop at work runs Windows XP, but my laptop is a MacBook Pro.  As a result, I only use OneNote when I’m at my desk.  Unfortunately, I find that my most critical note-inducing events happen when I’m on the go.  I tend to have my best thoughts when I’m on the move, and people frequently ask me to do things when we’re away from the desk.

On the other hand, Evernote is free and platform agnostic.  I have Evernote installed on my laptop, desktop, and iPhone.  I can also use it from the web.  Evernote keeps all of these clients sychronized.  I’ve never had a problem with conflicts, altough I’ve never tested modifying the same note from two different computers.  The search is speedy and the interface is clean.

I prefer Evernote’s continuous scrolling log as opposed to OneNote’s folders and pages.  In OneNote I wasted too much time fretting over whether I had properly categorized a note.  I also fiddled too much with the placement of notes on a OneNote page (they can be placed anywhere on the screen, whereas Evernote doesn’t have that capability).

One annoyance in Evernote is how it assigns the title for a new note in the Windows client.  The default title is the first setence of the note’s body, which is redudant.  I’d prefer something more clever, like the first few key words.  Either that, or I’d prefer a different entry flow.  The OS X client has an entry flow similar to composing an email.  When you create a new note you start entering the title and tags before entering the content area.

Also, the Evernote Windows client always has a new empty note sitting at the bottom of the journal, which I find a bit confusing.  It’s like always having a new, blank email message open in your mail client.  In general, the OS X client is superior to the Windows client.

Although Evernote supports synchronization at its core, it doesn’t have collaboration cababilities.  It allows you to publish notebooks to the web, but that doesn’t interest me.  I want to collaborate on a shared notebook with other Evernote users.  Essentially, I want synchronization between Evernote users.  I suppose I could give other people access to my Evernote account, but that’s clearly flawed.  Also, when collaborating with other users, synchronization would need to be nearly realtime instead of on demand or every 30 minutes or so.

Lastly, I’d like to see Evernote support tasks a little better.  You can insert checkboxes into your notes, but I’d like to be able to highlight a sentence and explicitly tag it as a todo task.  Then I’d like to be able to add metadata such as who it’s assigned to and when it’s due.

Overall, I have a great first impression of Evernote.  I’m curious to see how it scales up to many notes and to see how it changes over time.

Leave a Comment

Follow

Get every new post delivered to your Inbox.