After much discussion of what goes on below the water line (in the provider/enabler space), hopefully this prototype release will refocus energy and get the creative juices flowing above it (with the users themselves). The idea is to follow Unix’s example (as explained below) by seamlessly blending cloud resources into existing working environments. See examples below for some ideas of what is possible – my favourite is taking a local file and pipelining it through two web based services (the first is a rudimentary spam filter and the second appends legal notices), but being able to interact with HTTP in general from the command line is quite cool. Support for true bidirectional pipes (eg via CONNECT rather than simplex POSTs) and remote instantiation of pipelines is… well… in the pipeline. As is dealing with content types (verbs and nouns having already been catered for), mapping arguments to the query string and a flexible authentication infrastructure (all of which will require some thought and creation of static interfaces).
It follows the wildly successful Unix philosophy:
- Write programs components that do one thing and do it well.
- Write programs components to work together.
- Write programs components to handle text HTTP streams, because that is a universal interface.
- Verbs (constrained) e.g. GET
- Nouns (unconstrained) e.g. URIs (https://samj.net/])
- Content Types (constrained, optional */*) e.g. HTML (text/html)
To install the python prototype cush.py
- Change to a suitable directory (eg /usr/local/bin) and be sure you can write (eg sudo -s)
- If you have Subversion do svn export http://cush.googlecode.com/svn/trunk/src/python/cush.py so the keyword substitutions work
- Otherwise download http://cush.googlecode.com/svn/trunk/src/python/cush.py:
- curl -O http://cush.googlecode.com/svn/trunk/src/python/cush.py or
- wget http://cush.googlecode.com/svn/trunk/src/python/cush.py
- Make the file executable with chmod 755 cush.py
- cush.py install to create symlinks for the commands
- Download http://cush.googlecode.com/svn/trunk/src/python/cush.py (right click, Save As)
- cush.py upgrade periodically to stay up-to-date
Usage: cush [verb] [noun] or: [verb] [noun] eg: get http://samj.net
Cloud User SHell (cush) is a multi-call binary (like BusyBox) that combines
many useful cloud computing utilities into a single executable. Most people
will create a link to cush for each function they wish to use and cush will
act like whatever it was invoked as, but it can also be called directly and
passed the command as the first parameter.
Currently defined functions:
@, @@, delete, get, head, ls, post, propfind, put, rm
All of these work alongside your favourite shell, eg bash
Send files to/from the cloud and process them as if they were local:
$ echo "The quick brown fox jumps over the lazy dog." | put http://local/uploads/brown.txt $ get http://localhost/uploads/brown.txt The quick brown fox jumps over the lazy dog.
Even when they are:
$ get http://local/uploads/red.txt > /tmp/red.txt
$ get file:///tmp/red.txt
The quick red fox jumps over the lazy dog.
Dereference URIs with shorthand syntax (equivalent to GET command):
$ @ http://localhost/uploads/brown.txt
The quick brown fox jumps over the lazy dog.
Manipulate data in the cloud by listing (ls) and removing (rm) URIs like local files:
$ ls http://localhost/uploads/colours/
Set up HttpPipelines (like Unix pipelines) for data processing:
The following shows an rfc822 email message being pipelined through two URIs for spam filtering (which adds a X-Spam-Flag: YES header if it finds ‘Viagra’) and a legal notice (which appends ‘This message is confidential.’).
$ cat rfc822-spam.txt |@ http://localhost/cgi-bin/spam-vapouriser.cgi |@ http://localhost/cgi-bin/legal-notifier.cgi
Received: by 10.150.96.14 with HTTP; Tue, 5 Aug 2008 04:52:14 -0700 (PDT)
Content-Type: text/plain; charset=ISO-8859-1
Check out Viagra – it rocks.
This message is confidential.
Obtain and manipulate structured data (eg titles of the items in a feed, weather in Sydney):
$ get http://feeds.feedburner.com/samj | xpath //rss/channel/item/title
Found 24 nodes:
— NODE —
<title>Cloud standards: not so fast…</title>– NODE —
<title>Cloud computing and Open Source software</title>– NODE —
$ get http://weather.yahooapis.com/forecastrss?p=ASXX0112 | xpath /rss/channel/item/yweather:condition
Found 1 nodes:
— NODE —
Obtain information about URIs and technical debugging:
$ head https://samj.net/ content-length: 0 server: GFE/1.3 last-modified: Mon, 25 Aug 2008 22:04:36 GMT etag: "55ed4707-7cbc-4208-9cbc-cbc6105bc682"
cache-control: max-age=0 private
date: Thu, 28 Aug 2008 00:22:45 GMT
content-type: text/html; charset=UTF-8