tech guy in midtown

Back in October I started playing with Amazon’s elastic compute cloud.  My basic goal was to do some distributed computing with Python.  Thanks to Amazon’s documentation and helpful writeups by some early adopters, I was able to get my environment setup and to get my head around the various command line incantations.  I quickly grew tired of repeatedly having to input the image and instance IDs on the command line.

I wanted a pythonic way of starting up instances, querying their status, and doing simple things like running a command on all of my instances.  I figured a pythonic EC2 framework would make it easier for me to interface with whatever distributed computing framework I chose.  Also, I like python.

I wrote a class, EC2Controller, that uses Amazon’s Ec2.py module to simplify several common operations.  I’ve posted the code to a Google Code project, ec2python, which can be found at http://code.google.com/p/ec2python/.  I suspect some or all of the functionality is implemented in the boto project, but I haven’t taken the time to investigate.

Here’s an example of how I use EC2Controller from ipython:

In [1]: import EC2Controller as EC2

In [2]: ec2 = EC2.EC2Controller(cluster_size=2, verbose=True)

In [3]: ec2.getInstances()
Out[3]: []

In [4]: ec2.startCluster()
startCluster.  Query for the current instances.
Starting 2 instances...

In [5]: ec2.getInstances()
Out[5]:
[
EC2Instance( instance ID: i-d73389be
             state:       pending
             image:       ami-ee2eca87
             public  DNS:
             private DNS:  ),

EC2Instance( instance ID: i-d63389bf
             state:       pending
             image:       ami-ee2eca87
             public  DNS:
             private DNS:  )]

In [6]: ec2.waitUntilAllAreRunning()
Out[6]: True

In [7]: ec2.getInstances()
Out[7]:
[
EC2Instance( instance ID: i-d73389be
             state:       running
             image:       ami-ee2eca87
             public  DNS: ec2-75-101-212-146.compute-1.amazonaws.com
             private DNS: domU-12-31-39-00-54-E1.compute-1.internal ),

EC2Instance( instance ID: i-d63389bf
             state:       running
             image:       ami-ee2eca87
             public  DNS: ec2-174-129-150-216.compute-1.amazonaws.com
             private DNS: domU-12-31-39-00-B0-67.compute-1.internal )]

In [8]: ec2.stopCluster()
stopCluster.  Query for the current instances.
Terminate these instances: '['i-d73389be', 'i-d63389bf']'

I hope somebody finds the code useful.