CL-EC2 is a Common Lisp library for communicating with Amazon's EC2 and CloudWatch Query APIs. It was originally written as a foundation for Saffron Technology's Lisp-based Sierra management environment. As such, CL-EC2 does not implement the entire EC2 Query API, but rather a fairly large subset that may be used to manipulate/query images, instances, volumes, snapshots, etc. In other words, most of the useful Query API is present, at least from my perspective, but more should be included to finish off the entire API.
So far, CL-EC2 has only been tested on SBCL (I use version 1.0.30). There is a minor implementation dependency (for retrieving environment variables) that has been abstracted, so porting CL-EC2 itself should be very easy. However, the library does have dependencies on a few other open-source packages that must be supported by your Lisp platform of choice:
If I recall, I was able to use ASDF-INSTALL with most (if not all) of these, using SBCL. YMMV with other Lisp implementations.
CL-EC2 is released under an MIT-style license.
Right now, generally the source code is the documentation. If you look at api.lisp and cloudwatch.lisp you'll see functions that mimic, in a Lispy fashion, those aspects of the EC2 Query API supported by CL-EC2. As the project evolves documentation will improve.
To help get you started, below is a code fragment from our Sierra environment that uses the most complicated CL-EC2 interface -- running an AMI instance. Once you've seen the code in api.lisp you'll see these operations are straightforward. The only thing to note here is that the :USER-DATA keyword parameter must be a hash table that represents key/value pairs; EC2:RUN-INSTANCES takes this table and transforms it into a comma-separated list of these pairs. I have no illusions that this approach is ideal; it simply reflects the needs of our Sierra environment, and the fact that the library was not originally intended for general consumption. This is one of probably several weak areas in the API that will change as the project evolves.
(defun start-instance (machine user-data)
(ec2:run-instances (get-ami-id machine) (default-keyfile) :virtual-name (get-hostname machine)
:instance-type (get-type machine) :user-data user-data :monitor-instance (get-monitoring machine)))
I apologize for the lack of written documentation, especially code examples. As the project moves forward I'll try and correct this.
There are two environment variables you'll need to set before using the library: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. If you've any familiarity at all with EC2 you'll know what these are. Optionally, you may also set AWS_DEFAULT_AVAILABILITY_ZONE to one of the valid EC2 availability zones, if you typically use just one (as we do right now).
The current version of CL-EC2 uses Signature Version 2 for its query signing. There is also an implementation of Version 1; however, Amazon has apparently deprecated this version and they claim at some point it will no longer be supported.
Here are a few examples taken from our Sierra environment, which was written atop CL-EC2. Note that Sierra-specific functions are not included in CL-EC2.
SIERRA(36): (ec2:describe-images)
(#<EC2:AMI id: ami-2b16fa42 (sierra-devel-20100217/sierra-devel-20100217.manifest.xml) {1004647241}>
#<EC2:AMI id: ami-411df128 (sierra-devel-01142010/sierra-devel-01142010.manifest.xml) {10046473A1}>
#<EC2:AMI id: ami-6904e900 (sierra-devel/sierra-devel.manifest.xml) {1004647501}>
#<EC2:AMI id: ami-9c17f5f5 (amismb-centos53/amismb-centos53.manifest.xml) {1004647661}>)
SIERRA(37):
SIERRA(38): (ec2:describe-instances)
(#<EC2:AMI-INSTANCE i-2710ee4c; ami: ami-2b16fa42; state: running; dns: ec2-00-000-00-000.compute-1.amazonaws.com {10046CF6D1}>
#<EC2:AMI-INSTANCE i-3511ef5e; ami: ami-2b16fa42; state: running; dns: ec2-00-000-000-000.compute-1.amazonaws.com {10046CFC81}>)
SIERRA(39):
SIERRA(40): (ec2:describe-volumes)
(#<EC2:VOLUME id: vol-de824200; zone: us-east-1d; status: in-use {1004644571}>
#<EC2:VOLUME id: vol-819b6100; zone: us-east-1d; status: available {10046446D1}>
#<EC2:VOLUME id: vol-9339f800; zone: us-east-1d; status: available {1004644831}>
SIERRA(41):
SIERRA(42): (show-stats (cw:get-metric-statistics "NetworkOut" "2010-02-01T00:00:00" "2010-03-01T00:00:00"
:period 3600 :statistics '("Sum" "Minimum" "Maximum")))
TIME SAMPLES AVERAGE SUM MIN MAX UNIT
2010-02-24T14:00:00Z 120.00 n/a 1158.0000 0.0000 353.0000 Bytes
2010-02-24T13:00:00Z 98.00 n/a 1648388.0000 0.0000 880308.0000 Bytes
2010-02-24T15:00:00Z 32.00 n/a 8312.0000 0.0000 2552.0000 Bytes
SIERRA(43):
SIERRA(44): (start-sierra)
(#<EC2:INITIATED-INSTANCE i-8d7684e6 (solace); reservation: r-f8ca8f90 {1004E5ACC1}>
SIERRA(45):
SIERRA(46): (ping)
(("solace" . "pending"))
SIERRA(47):
SIERRA(48): (ping)
(("solace" . "ec2-00-000-00-000.compute-1.amazonaws.com")
("grace" . "ec2-00-000-000-000.compute-1.amazonaws.com"))
SIERRA(49):
You can download the source code as a gzip'd tarball here. Use ASDF to build/install it.
I would be thrilled to accept contributions from users of CL-EC2. There are aspects of the EC2 API that are unimplemented because I've had no need for them; current function interfaces are likely not ideal for the general community; and there are probably bugs I haven't encountered yet. At some point, if contributions begin coming in, I'll get the source code into Subversion.
SaffronSierra is a developer Platform-as-a-Service for Saffron Technology's Natural Intelligence Platform, including SaffronMemoryBase. Using Associative Memory technology, SaffronMemoryBase simplifies data analysis and helps uncover the "World of Experience" that lies within different kinds of data. By leveraging the SaffronMemoryBase REST APIs, it's possible to quickly find connections among people, places, things and events in data, along with their frequency counts.
2010-02-25 David E. Young
* Release 0.3.
* Added some additional slots to AMI-INSTANCE (architecture, private and public ip addresses).
* Added some examples to the web page.
2010-02-24 David E. Young
* Release 0.2.
* Borrowed Drakma's URL-ENCODE function and modified it to conform to the Amazon Version 2 signing specification by encoding
spaces as %20 rather than '+'.
2010-02-23 David E. Young
* Release 0.1.
* Implements EC2 API version "2009-11-30".
* Implements CloudWatch API version "2009-05-15".