Posts Tagged Business

Climbing Two Fourteeners in Colorado

September 7, 2009 at 8:05 pm · Filed under Random Thoughts ·Tagged ,

Yesterday I hiked to the 14,271 foot peak of Mt Quandary.  Quandary Peak was my second fourteener.  A couple months ago, on July 4th, I summited Gray’s Peak.

Hiking reference guides declare that both hikes are easy by fourteener standards.  The key phrase there is “fourteener standards,” because they’re difficult.  Granted, I did both hikes just one day after flying in from New York, so I wasn’t acclimated to the altitude.  Even so, with a pack on your back and loose rocks under your feet, it’s always tough going past 13,000 feet.

Catching my breath on Mt Quandary's peak

Catching my breath on Mt Quandary's peak

The hike provides a lot of metaphors for business and software development.

  1. Ultimately, you only reach the top by repeatedly putting one foot in front of the other.  To ensure success, break down the journey into a series of short term, easy to attain goals.  That’s how you get things done.
  2. Experience helps.  On my first climb, even though I had read some trip reports, I was surprised at how difficult it was to keep moving in high altitudes.  I had to take breaks every 10 feet.  I wondered if the summit was worth the effort.  On the second hike, I expected to go slow, and I knew the summit was an ample reward.
  3. Although the ascent and summit contain all the glory, the descent is both more difficult and equally important.  It’s exhilarating to release a new product or major feature, but you have to be prepared to maintain it and complete the life cycle.
  4. If you need to remember part of your journey, then you have to document it.  On both descents there were several points where I had egregious recall of key landmarks.  For example, I’d think “wow, I really thought the tree line was much closer to this big white rock.  I can’t even see the tree line from here!”  Fortunately trails were well marked, so this wasn’t a danger.  In business, you must document key problems and decisions.  Otherwise your mind will surely blur them out over time.  If the trail isn’t well marked, you’ll get lost.
  5. You can find excitement in every stage in the journey, from shopping for supplies, to standing at the peak, to tasting a celebratory beer.

Comments (1)

Pokerbot Lesson: Work Hard During the Good Times

August 26, 2009 at 9:03 pm · Filed under Random Thoughts ·Tagged , , ,

As I mentioned in an earlier post, I wrote a poker bot in 2004.  I learned a lot from that project, and I’d like to share some of the lessons learned over a series of blog posts.

All poker players are familiar with the game’s swings.  Poker’s unpredictable sequence of wins and losses stems from the addictive cocktail of uncertainty and randomness that makes the game so popular.  There are many lessons to learn while riding the roller coaster of wins and losses, but I’ll focus on one that’s particular to a poker bot author (or an investor in the markets).

Imagine if you unleashed your poker bot algorithm and observed the following performance over the first 10,000 hands.

Pokerbot performance over 10,000 hands

Pokerbot performance over 10,000 hands

It takes a poker bot about 22 hours to play 10,000 hands.  This assumes that the bot plays 10 tables simultaneously, averaging 45 hands per hour at each table.  For comparison, it takes a human about 28 days to play 10,000 hands, assuming he plays an intense 30 hands an hour, 12 hours a day.  Although it’s not a large enough sample to yield a statistically significant win rate, it’s a sizable sample.

The first couple of times that this happened to me, I couldn’t help myself from leaning back and thinking that my troubles were over.  Instead of analyzing more data or writing more code, I’d relax.  Then I’d see this turn of fortune.

Fictional graph of big bets won or loss over time.

Fictional graph of big bets won or loss over time.

The downturn would give me a kick in the pants, and I’d dive in again to my work with vigor.

After this pattern happened a couple times, I realized how much time I was wasting.  Frustrated by this inefficient cycle, I made a point of staying vigilant even when the results looked promising.  I learned that you have to work hard even when everything’s going smoothly.

This lesson is applicable to both investing and software development.  When your PnL is positive, you need to make an extra effort to stay sharp.  When you’re writing software, you can’t assume that your program is correct just because it runs without throwing off errors.  Be suspicious and stay vigilant, even when things seem okay.

Comments (2)

Navigating New York’s Attitude toward Technologists

August 17, 2009 at 5:52 pm · Filed under Business, Random Thoughts ·Tagged , , , , ,

I just read Zed Shaw’s post where he announces that he found a job in San Francisco.  Congratulations to him.  In his post, I appreciated his concluding comparison of how companies in New York and San Fran perceive and approach technologists.  Zed says:

NYC prospects were looking for a badass/ninja/rockstar beta-male “techie” employee to make them rich creating lame applications for giant Finance/Fashion/Marketing companies.  SF prospects were looking for a partner to get rich with them creating great products for customers.

To a large extent, I agree.  New York is filled with companies that have non-technical core competencies.  These businesses treat technology as a necessary cost.  When they search for a rockstar ninja, their goal is to minimize cost.  The thinking is “that last tech we had was too slow and too sloppy, and his mistakes cost us money.  We need a superstar tech guy to come in here and do things right (not screw up).”

The dynamic is top-down.  The business wants a super star who will take orders and work efficiently.  On the whole, this makes perfect sense.  As Judge Smails says, “the world needs ditch diggers, too.”

This system fails for superstar techies like Zed Shaw.  This class of techies are often prima donna’s in the same vein as star wide receivers in the NFL.  Instead of taking orders, they want to provide advice.  Instead minimizing cost, they want to generate value.  Indeed, they want to be partners, not employees.  (Even if the company pays well, as many New York firms do).

San Francisco, on the other hand, is a town filled with ninjas looking to band together.  Each one is a Steve Jobs looking for a Steve Wozniak to join them in the garage.  This is a natural and attractive situation, as evidenced by Zed’s move from his beloved LES.

New Yorkers do have hope.  The key is to pursue and promote every opportunity to increase revenue instead of decreasing costs.  When you see this opportunities, you need to explicitly point them out.  Then, if you’re given the green light, work extra hard to execute.  If you score a couple wins, then you can take a partnership role (even in New York).

Comments (1)

Maker’s Schedule in Finance

July 29, 2009 at 10:03 pm · Filed under Business ·Tagged ,

I just read the Freakonomics article responding to Paul Graham’s recent essay, “Maker’s Scheduler, Manager’s Schedule.”  The short summary is that managers break their day into 1 hour blocks and makers (programmers, writers, etc) break their day into halves.  Therefore a one hour meeting can destroy half of a maker’s day.

I’m certainly a maker who’s usually pessimistic about the utility of meetings.  I know the productive rush of putting on my headphones and diving head first into a program.  I’ve worked that way before.  It’s nice, and it works for me.

But I work at a startup in finance.  It is simply not possible to structure my day to avoid meetings or interruptions.  In particular:

  1. You can’t work all night and saunter in at 11am the next day.  The market is open from 9:30 to 4:00, so you absolutely must be there for those hours.  Even if you don’t personally interact with the market, it’s against the culture to be absent during trading hours.  If you want to get ahead and make a difference, you need to be there early and leave late.
  2. The financial world is fast-paced and event-driven.  You can’t go dark for hours or days on end.  When something happens, people around you react, and winners jump up and pitch in.
  3. For technologists in finance, the customer is king.  The “customer” might be an investor, trader, or boss.  If they want to talk, you respond immediately.  An attitude of “hey, I’m knee deep in some code” will not work.

These realities mean you either need to adapt or fail.  Here are some of my techniques for producing quality software within the fast-paced, event-driven world of finance.

  1. I use OmniFocus to manage my tasks.  I’ve tried other task management programs before, but none of them work as well as OmniFocus.  It encourages me to target small, bite-sized tasks that are robust to interruptions.
  2. I put off deep dives to the evening or weekends.  I think it’s counterproductive to make a habit of long hours or working on weekends, but it’s sometimes necessary.
  3. I take note of common requests, and find time to write a program to help me fulfill them, even if the program is an undocumented script that’s for my eyes only.  This reduces the severity of interruptions.
  4. Be blunt about ending meetings.  You can’t always be Mr. Nice Guy.  Sometimes you just have to cut somebody off and get back to work.

I’d love to hear how other makers manage the reality of their work environment.

Comments (1)

Automate Everything for Peace of Mind

July 28, 2009 at 5:42 pm · Filed under software ·Tagged , , ,

Anybody who has written a computer program appreciates the warm glow of having a machine do your work.  That pleasure often masks the burden of holding the program’s hand — kicking it off and checking that it ran correctly.

At least for me, the natural tendency is to stop short of complete automation.  Instead, if I’m being lazy, I’ll start the program manually and catch errors by noticing odd behavior.  Perhaps I’ll see a glaring failure message.  Or maybe I’ll just notice that the program ran too quickly.

I do this because I’ve seen trusty programs fail in myriad ways.  I’ve seen malformed or truncated inputs.  I’ve seen choppy or slow network connections.  I’ve seen disks fill up.  I’ve seen memory hogs.

When an error like that pops up when you’re sitting right there at the command prompt, it’s typically easy to diagnose and move on.  When run naively from a scheduler, you might not notice a problem until disaster strikes, sparking a late night emergency call, or worse.  Therefore, I have to fight my natural tendency to hand-hold my programs.

Through experience, I’ve learned to go the extra mile and properly automate my programs.  These are some of my best practices — I hope other people can point out ones that I’ve overlooked.

  1. Every program must use a proper logger.  The first thing I do when I write *any* program is to make sure it’s set up to log using that language’s preferred logging utility.  This allows you to properly investigate what happened every time the program runs.
  2. Save the logs from each run.  It’s possible that an error will go unnoticed for a while, and there’s nothing worse than saying “unfortunately the log file was deleted.”
  3. Validate the program’s inputs.  I’ve been burned before by people who will “always save their Excel spreadsheet in the same way,” “websites that never change,” and data providers that “guarantee their data are validated before given to you.”
  4. Sanity check the outputs.  If you always expect your program to generate some new data, then compare the new results to the old results, and verify that there are actually new results.
  5. Catch errors and retry.  Computers are deterministic, but real world resources are unpredictable. I’ve seen one-off errors too often.  Perhaps I call an external program that fails for an unknown reason.  Or a file copy fails, but when I retry, it works fine.
  6. Have an extremely robust email notification system.  When there’s an error, you should be notified.  One trick is that a failure in the email system should never cause the program to fail.  Perfect this library and reuse it.
  7. Try your program every time you change it.  This is easy to say, but it’s tempting to say “oh, but I’m just adding one line” and not run your unit tests.  If you change it, at the very least, give it a whirl.
  8. Run the program from multiple machines.  Computers fail at all the wrong times.  At the very least, design your program so that only one instance publishes a final result.  Then if that computer fails, you can run a quick command on the other machine to publish whatever that program creates.

Leave a Comment

SPY Closing Price Update

July 20, 2009 at 9:45 am · Filed under Business ·Tagged , , , ,

Just to round out my quick post on dirty financial data, I came into work today and saw that Thursday’s closing price for SPY is now correctly stated as 94.15.  Sometimes I half-wonder if somebody intentionally causes these mistakes just to put a stick in the wheel of quantitative backtests.

spy_hp_20090720

Leave a Comment

Dirty Financial Data: SPY Closing Price

July 17, 2009 at 1:29 pm · Filed under Business ·Tagged , , , ,

Before joining finance, my naive assumption was that the market’s high stakes would necessitate accurate, high quality data.  In particular, I expected frequently traded stocks and ETFs on public exchanges to have accurately quoted end of day prices.  In reality, even those data are noisy.

Yesterday’s (7/16/2009) closing price for SPY is one example.  The NYSE reported a composite closing price of 93.11 for SPY, but the intraday price graph makes it clear that 94.15 is much more accurate.  Here are two Bloomberg screens showing both the tabulated closing price and the intraday price plot.

spy_hp_20090717

spy_gip2_20090717

Over 200 million shares of SPY trade every business day, yet bad closing prices like this still pop up.

Leave a Comment

Scatter Map of Madoff’s Victims

February 6, 2009 at 7:03 pm · Filed under Python, Random Thoughts ·Tagged , , , ,

When I learned that the court had made a list of Madoff’s victims available, I immediately wanted to graph it.  I first forwarded the PDF one of my best friends.  Within a couple hours he had converted it to text, which he streamed through Yahoo Pipes to generate KML suitable for viewing with Google Maps.  The pipes filtered out duplicates, did address discovery and geolocation.  Unfortunately, Google Maps can’t display more than 80 markers at once, which is far less than the thousands of people whom Madoff ripped off.  He eventually did pull off a single view visualization, but it ran like a dog.

So last night I took a different approach.  I extracted out all of the zip codes from the list.  (Ignoring all of the international investors robbed by Madoff.)  Then I used http://geocoder.us to map each zip code to a latitude and longitude.  Next I wrote a short script in NodeBox to draw a projection of the zip codes on a canvas.  Each point is color coded to indicate the number of times that zip code occurs in the court document.  I was helped a lot by code and ideas in Ben Fry’s Visualizing Data book, which I have access to via Safari Bookshelf.  In particular, I stole his function for projecting out the latitude and longitude numbers.

Here are the resulting images.  The perfectionist in me would love to tinker with this for hours.

scatter map of Madoff's victims in the NYC Metro area

scatter map of Madoff's victims in the NYC Metro area

scatter map of Madoff's victims in NYC Metro area

scatter map of Madoff's victims in NYC Metro area

scatter map of Madoff's victims in New England

scatter map of Madoff's victims in New England

scatter map of Madoff's victims in Florida

scatter map of Madoff's victims in Florida

Comments (6)

You Can’t Cheat an Honest Man

January 27, 2009 at 4:57 pm · Filed under Business ·Tagged ,

There was a long investigative reporting piece on Bloomberg today about Bernie Madoff’s enablers.  The article makes the point that many of Bernie’s investors and feeders suspected Bernie was a cheat, but they didn’t mind as long as they benefited.  For example, it quotes a fund of fund’s executive as responding to an accusation that Madoff was front-running his brokerage clients by saying, “Yeah, so what?  That’s his edge.”

The article mentions a book, by James Walsh, titled You Can’t Cheat an Honest Man.  I like the title, which he borrowed from a 1939 W.C. Fields movie.  Although I feel badly for the people whom Madoff victimized, I think that the phrase rings true.  For years people had been claiming that Madoff’s returns weren’t feasible, but investors piled on anyway.  Some feeders, such as Fairfield Greenwich, “put 53 percent of its assets into Madoff’s hands and charged clients 20 percent of profits; it added a 1 percent management fee in October 2004.  For the $7.3 billion funneled to Madoff, they reaped $102 million per year, assuming Madoff returned a phantom 10%.

Not all people went along for the ride though.  The article reports that Oswald Gruebel, who was head of private banking at Credit Suisse, suspected something was wrong and “urged customers to withdraw from Madoff’s funds.”  Apparently only about half of the money was taken out.

Incidentally, a google search for the phrase, “you can’t cheat an honest man” turned up this prescient editorial from November, 2005 by Christopher Laird.  He wrote:

The US right now is in the same economic boat. A 5 year housing bubble has caused real estate to rise in most cites by over 100%, and because real estate is easy to leverage, people using historically low interest rates have basically succeeded in mortgaging their houses well past levels that would be sustainable.

Now people have been enticed into an endless cycle of buying on credit cards and sold the lie of using a home mortgage to pay those off, only to begin the cycle again. And now we have these new bankruptcy laws that are waiting to lower the boom on the many people who were enticed and unwisely accumulated lots of debts, having been seduced by a chimera of rising housing values that are only going to fall off a cliff once the people realize the jig is up. 90% of them will lose their shirts… sadly….

Leave a Comment

My New Favorite Interview Question

June 30, 2008 at 8:30 pm · Filed under Business ·Tagged ,

Lately I’ve been doing a lot of interviews, so I’ve been reflecting on the effectiveness of my questions and my methodology. I generally take a relaxed, conversational approach to interviewing. I consider myself to be a strong judge of character and a well-rounded conversationalist, so I like to get the interviewee talking and see where things go. My goal is to let the interviewee do most of the talking, prompted by me to guide the topic or curtail the time.

One of the biggest challenges in an interview is calming the applicant down so that they behave normally. I know interviews can be overwhelming — especially for technologists who sometimes are a bit weak on social skills. After a general chat about my company and their background, I used to ask an extremely simple programming question. For example, I might ask them to write a function to print out a multiplication table.

A surprising number of people stumble on this initial question. Often times I think it’s due to nerves. So now I’ve started to ask the following question:

To start us off drilling down into your skills, I’d like to start with a topic that you’re extremely comfortable with. Please think of a question, and then answer your own question. I’d like you to first clearly define what the question is. Then walk me through the answer. Try to keep it brief.

I think this is a great question for several reasons:

  1. It keeps them talking.
  2. I stay interested because I often learn something new. Even bad interviews become a better use of my time.
  3. What they choose as the question is a unique indicator of their interests and talents.
  4. Do they follow instructions? A surprising number of candidates fail to define a question. Instead he or she simply launches into a monologue.
  5. Can she answer her own question? A surprising number of candidates get tripped up answering their own question.  That’s bad.
  6. Does he have a sense of humor? It’s a great opening for the candidate to make some sort of joke because this is a fairly bizarre question. One candidate quipped, “you want me to interview myself?”

Many times I will have to cut a candidate off during their answer, but that’s okay. I typically use it as a nice springboard into a question of my choosing with a segue such as “You mentioned randomization. Could you write me a function that accepts a list of integers and returns a new list with the integers shuffled?”

Leave a Comment

Older Posts »
Follow

Get every new post delivered to your Inbox.