Osh

In a Unix shell, you can run a series of commands, connecting the output of one command with the input to the next command using pipes. The information passed from one command to the next is normally organized into lines of text. Tools such as awk and sed are designed to extract and transform information from these lines. It would be much simpler if the data passed between commands was organized into objects. Osh is based on this idea. Osh is implemented in Python, so much of osh will be familiar if you already know Python.

Osh is not a shell. It is an executable implementing a language that supports the composition of commands through piping. The objects piped from one command to the next can be Python primitives (numbers, lists, maps, etc.), other useful types such as dates/times, or database rows; or represent various OS resources such as files, directories, and processes. Conversions between objects and strings simplify integration with the Unix environment. The commands included with osh manipulate objects, access databases, and execute commands remotely, including parallel execution on nodes in a cluster. In a single osh invocation, all commands run in a single process. Multithreading is used only when executing commands on a cluster.

Example: Suppose you have a cluster named fred, with nodes fred1, fred2, fred3. Each node has a database tracking work requests with a table named request. You can find the total number of open requests in each database as follows:

[jao@zack] osh @fred [ sql "select count(*) from request where state = 'open'" ] ^ out
('fred1', 1)
('fred2', 0)
('fred3', 5)

Example, continued: Now suppose you want to find the total number of open requests across the cluster. You can pipe the tuples into an aggregation command:

[jao@zack] osh @fred [ sql "select count(*) from request where state = 'open'" ] ^ agg 0 'total, node, count: total + count' $
6
More information:

License: GPL
Release history
Tutorial (TBD)
User guide
Download
Software with similar goals
PyCon 2006 paper on osh
PyCon 2006 talk on osh
jao@geophile.com