Lily

My Arduino arrived yesterday and I jumped straight into getting it talking to Lily. Flash is able to interface with the Arduino using XMLSocket to communicate via a socket/serial proxy, but native Javascript doesn't offer anything like XMLSocket. Fortunately, Mozilla exposes a scriptable socket interface that's close enough for what we want to do.

It took a couple of hours to get it up and running, using the socket-transport-service to send messages to a java serial proxy and from there to the Arduino. On the Arduino I used SimpleMessageSystem to handle receving and sending messages. SMS is specifically intended to work with message passing programs like Max/MSP & PD, so it's a good fit for Lily. The Lily SimpleMessageSystem object is available here

Next up- removing the need for serial proxying and talking (via javaxpcom) directly to the serial port from Lily.

Lily screencast showing building a simple patch that adds two numbers. Demonstrates patch editing techniques, using arguments to pass a paramter into an object at initialization, message ordering (when multiple connections are made to one outlet, messages are output in the order the connections were made) and using feedback by passing a process's output back into its input via a delay.

Demo screencast of a rudimentary beatbox created in Lily. Probably not a use for which the browser is best suited but hopefully gives a sense of what's possible. In any event it cracks me up. Patch can be downloaded here.

Demonstration of the help patch for the circle object. Help patches are small programs that document the available commands for an object. They're accessible by option/alt clicking on an object or via the Help menu item. Because a help patch is a functioning program, it's possible to copy working code directly from the help patch into your current program.

Two minute screencast of building a patch to fetch Flickr "interestingness" pics for a given day. Demonstrates how web APIs and 3rd party framework JS can be wrapped as Lily objects- in this case, the flickr.interestingness.getList service, the Yahoo YUI Calendar widget and Lokesh Dhakar's Lightbox script. Also shows how a Lily patch can contain other patches (note the use here of the "format-date.json" patch loaded using the patcher object) which makes it possible to encapsulate and reuse patches.

Lily patch building screencast demonstrating reading a text file from disk (using the text object) and then displaying its contents- first as a slideshow (using marquee) and then as a series of randomly sized/positioned comments. The demo also shows how to hide objects and connections.

Demo showing how to combine web services to perform complex operations- in this patch, the NY Times homepage RSS feed is fetched using the rss object, the descriptions are concatenated and passed to the Yahoo term extraction API. The extracted terms are then passed to the Flickr search API and the resulting photos and captions are displayed in a table.

Feeble attempt at this processing demo. The patch uses a metro to output mouse position at regular intervals. Each time the position is output a circle object is created at that position and a reference to the circle is passed to the animate object which manipulates the object's opacity. When the opacity reaches zero, the circle's delete method is called.

Minimal web server. Incoming http requests are output through the outlet and the response is issued after the left inlet receives some input. The demo screencast shows how to use the httpd object to enable a web browser to control a patch remotely.

Here's a small Lily program that lets you check the spelling of a word from your cellphone- it uses the Twitter API to send and receive messages from the cellphone and the Yahoo Suggested Spelling web service to look up the spelling. To use it you'll need two Twitter accounts- one for Lily and one for yourself. You'll also need to enable your cellphone in the Twitter account settings. In the patch you'll need to add your Twitter user/pass as an argument to the twitter object.

Once you're setup, run the spellchecker by texting "d <your-lily-account-name> <word-to-check>" to 40404. You should receive an SMS back from Lily with the correct spelling shortly.

The new twitter object implements most of the api methods, including the direct message feature which I'm using in this patch. The code also includes an example of doing basic auth with xmlhttp. You can download it and the twitter spellchecker patch here.