Lily

Finally found a use for the NY Times from Bill Orcutt on Vimeo.

Ever wondered what a web site sounds like? Not the words, but the actual data on the site itself? Here's a Lily application that let's you "play" the data in a web page like an instrument.

When the patch starts, the browser enters a DOM inspection mode and mousing over a DOM element highlights the node. Clicking on a node writes the element's data (its innerHTML value if it's a text element or the binary data if it's an image) as a sound file and the file is then loaded in a quicktime player in the patch. The sounds can then be triggered using OSC messages. While a DOM element is "playing", the browser scrolls to and highlights the element with a thick black border.

In the video, I'm spazzing out on the NY Times homepage using the monome controller, but the demo should work with any OSC enabled controller. Sound conversion code based in part on a javascript port of the baudio project.