Lily

Devjavu, the company that was providing us with free project hosting, is closing so I've migrated everything to Google Code- the new url for the dev wiki, issue tracker and source code is http://code.google.com/p/lilyapp/

The SVN checkout command is now: "svn checkout http://lilyapp.googlecode.com/svn/trunk/ lily"

I think I've updated all the links, but since I'm probably forgetting something, please let me know if you find anything that's out of date.

From the beginning, Lily has used Firefox's SVG capabilities for various shape primitive externals ("circle", "rect", etc) and to draw connections between objects, but until this weekend, it lacked a straightforward way to use SVG's capabilities for animation and graphics. To address this, I've added a new UI object, "svg" that provides a resizable SVG canvas to draw to. As an added bonus, the drawing API is compatible with the Processing API, meaning that "svg" can run most of the 2-D examples on the Processing site with minimal modification. "svg" drawing commands can be messages from other objects sent via connections or run from an external script file. It's also possible for an external script to define a method that can be called with parameters from an input message. Scripts can also output messages from outlets.

As you'd expect with a Javascript-based SVG implementation, there are some limitations- there's no 3-D yet (though this could be added), no support for OpenGL, no reading & writing pixels directly, etc. There are performance limits as well, though this really only becomes an issue for animations that need to manipulate hundreds (or thousands) of objects many times a second. And with Tarmarin on the way, this should become less of an issue. There are even a couple of advantages- there's no compilation, so programs can be switched in and out instantly at runtime, and unlike an applet, SVG is completely transparent and interoperable with the web page it's embedded in. "svg" will be included in Beta 4 which should be ready sometime in next month. Interested in testing? Let us know.

A new Lily beta is available tonight- there's ton of bug fixes and some new features that open up possibilities not available in previous betas: there's a set of objects that allow a patch to interact with the main browser window a la Greasemonkey; a couple of web service externals to enable Lily programs to send and receive SMS & IM messages (via Twitter) or annotate web pages (thank you Shiftspace!); a couple of objects that implement the Open Sound Control protocol; and an extern that enables Lily to talk to the Arduino physical computing board. If you've signed up to test Lily previously, you should be receiving an email tonight with the new download url. We're still looking for testers, so if you're interested please get in touch.

Update: Security changes in Firefox 3 have broken the method described below. Java in components is still possible - see this thread on the SIMILE list - but using sockets to do interprocess communication between JS & Java is no longer possible AFAICT. OSC in Lily is now done using the Flosc project running as a background nsiProcess.

I checked in code this week to support Open Sound Control for Lily- it consists of two pieces: a Java component that implements methods from the JavaOSC library and two Javascript externals (oscsend & oscreceive) that use the OSC service.

Initially I assumed OSC could be done entirely in Javascript using nsISocketTransport- once I got into it, I quickly realized mozilla sockets don't support UDP. (incidentally, I bumped into this dormant project which has created UDP sockets, but apparently there are Mac build issues) Not being able to do it in JS meant I'd have to write an XPCOM component- something I'd manage to avoid so far. After a little poking around on the web, I decided against coding it in C++ simply because maintaining 3 separate builds (4 actually since the Mac Gecko SDK isn't universal) seemed too much like work. I looked at Python for about a second and then came around to Java.

Mozilla's 'official' project supporting Java components- JavaXPCOM- seems geared to embedding rather than extensions, but fortunately I stumbled across this from the folks at SIMILE which demonstrates how to load Java classes and call methods from extension code. Great, except I also needed to be able call into Javascript from Java, something the sample code doesn't do. After spending a silly amount of time trying to make it work with LiveConnect, I bagged it and just used sockets to do the Java to JavaScript communication, which works but is, like, totally suboptimal.

Having OSC opens up new ways of talking to Lily- with a helper Max patch, I'm able to use various game controllers as well as my new 40h. Also, now that I have OSC working, I'm planning on using what I've learned about Java components to add support for other fun stuff- serial access, Applescript, etc. I also have a couple of nice OSC Lily demos in the works I'll post here soon.

My Arduino arrived yesterday and I jumped straight into getting it talking to Lily. Flash is able to interface with the Arduino using XMLSocket to communicate via a socket/serial proxy, but native Javascript doesn't offer anything like XMLSocket. Fortunately, Mozilla exposes a scriptable socket interface that's close enough for what we want to do.

It took a couple of hours to get it up and running, using the socket-transport-service to send messages to a java serial proxy and from there to the Arduino. On the Arduino I used SimpleMessageSystem to handle receving and sending messages. SMS is specifically intended to work with message passing programs like Max/MSP & PD, so it's a good fit for Lily. The Lily SimpleMessageSystem object is available here

Next up- removing the need for serial proxying and talking (via javaxpcom) directly to the serial port from Lily.