Lily

The first Lily public beta is available for download- you can grab it here or via the now enabled download link in the header. This first beta is essentially feature complete for the 0.1 release. There's a new set of demo applications, a few new externals and the usual load of bug fixes, including some significant fixes for Linux. The complete changelog is available here. Download it, check out the new examples and demo applications, join the user group and get started making something of your own.

With the public beta finally out, I'll be focusing on bug fixes, improving the documentation and preparing the source for the move to a public repository. And of course, getting ready for Firefox 3.

Inspired by Marcel Salathé's processing applet, here's four svg/javascript sketches that visualize the HTML DOM in various ways. Each sketch walks the HTML document loaded in the browser and draws the structure in the patch window using the svg external's api.

Randomized, force-directed layout based on same traer physics example that seems to have been the basis for Marcel Salathé's work. Painfully slow and unstable (it seems to fly off the canvas almost as often as it produces a visible result), though the misfires are sometimes more interesting than the intended result. JS/SVG performance issues limit the usefulness of the script for larger sites- as more objects are added to the SVG DOM, the script slows until it grinds to a halt. Screenshot above generated from www.google.com.

Completely random layout inspired by the "network" illustration in Design By Numbers. Generated from sfbay.craigslist.org.

Examples of visualizing the DOM hierarchy as nested/stacked boxes. The outer/top box is the root node- child nodes are nested/stacked. Screenshots generated from www.google.com.

Drawing the DOM from Bill Orcutt on Vimeo.

Unlike some of the other Lily demos which have a putative utility, the CLI DOM Explorer is defiantly pointless. It's a command line DOM explorer that let's you telnet (!) to your browser and traverse & interact with the HTML document using standard Unix commands. The (somewhat) supported commands are: ls, pwd, cd & rm - ls displays the child nodes of the current node, cd changes the current node to the node supplied as an argument (".." & "/" are supported), pwd displays the path back to the root node and rm deletes the supplied node. The screencast more or less documents what you can do with it. (As with all the Lily demos, keep in mind that this is an alpha application made with beta software, so don't get your hopes up or anything.)

I'm a big fan of ffffound.com's image bookmarking site. Here's a local, client-side version of that service let's you archive images for offline viewing. It works by adding a "Save this image" item to your browser's context menu. Selecting the menu item while clicking on an image saves the binary image data and its context (the page url, date, etc) in browser storage. You can then view your saved images by pointing your browser at http://localhost:8080/archiver?offset=0.

Here's a Lily demo app that runs in the background while you surf the web. When you find something you'd like more information about, select the word(s), hit the enter key and page bot does an image search (using the flicker search api) and displays thumbnails of the results. Clicking the thumbnail displays a larger version of the image. Clicking the image again returns you to the thumbnails. Clicking away from the page bot dismisses it. Check out the screencast to get a feel for what it does.

This is a good place to mention that when you open a patch using "Open" in the Lily menu, it opens the patch as a separate window. This is fine for development, but if the patch doesn't have a UI (as in this example), then you'll probably want to keep the window hidden while you're running it. The way to do this is to save the patch as an addon and check "Hide Main Window" and "Hide Debug Window" in the save as dialog.

SVG to Quicktime from Bill Orcutt on Vimeo.

Here's a demo that uses an svg/javascript animation to control quicktime playback. The objects in the animation are attracted to the mouse and repelled by each other. The forces generated by their interactions are scaled and output to control mp3 playback rate. Based on this traer physics example. Features the music of Dillard Chandler .

SVG to OSC from Bill Orcutt on Vimeo.

Another svg/javascript animation to control sound. Each circle has a pitch/amplitude value based on its position. The values are sent (via the sendosc external) as OSC messages to a Chuck synthesis patch to generate the sound. Dragging the circles around modifies the harmony as the individual note values change. Running this demo requires Chuck. Based on this processing example by Sean McCullough.

SVG Midi Sequencer from Bill Orcutt on Vimeo.

Oddball monophonic sequencer using an svg/javascript animation. The animation generates a tree of randomly distributed nodes that represent the steps in the sequencer. Press the spacebar to add a node to the tree. Toggle notes on/off by clicking a node while holding down the "x" key. A node's note value is based on its position on the screen and is output as an OSC message to a Chuck patch that translates it to midi. Dragging the nodes around the screen in the tree changes their value. The sounds in the screencast are generated using Reaktor, but the demo should work with any midi enabled synth or sampler. Based on this traer physics example.

Finally found a use for the NY Times from Bill Orcutt on Vimeo.

Ever wondered what a web site sounds like? Not the words, but the actual data on the site itself? Here's a Lily application that let's you "play" the data in a web page like an instrument.

When the patch starts, the browser enters a DOM inspection mode and mousing over a DOM element highlights the node. Clicking on a node writes the element's data (its innerHTML value if it's a text element or the binary data if it's an image) as a sound file and the file is then loaded in a quicktime player in the patch. The sounds can then be triggered using OSC messages. While a DOM element is "playing", the browser scrolls to and highlights the element with a thick black border.

In the video, I'm spazzing out on the NY Times homepage using the monome controller, but the demo should work with any OSC enabled controller. Sound conversion code based in part on a javascript port of the baudio project.

Here's 3 two-minute screencasts that demonstrate building a local search mashup step by step in real-time. Best viewed in full-screen.

Performing a local search using the Yahoo local search api and displaying the results using a XUL tree widget. Using patcher to encapsulate patch code.

Adding a call to the Yahoo geocoding API and a Google map to display the location of the search.

Adding markers to the map for each item in the list.