Toshio Iwai’s Music Insects is singular for the seamless way in which it hybridizes a pixel-based paint program, a real-time system for composing and performing music, and a wholly visual programming environment for animated behaviors.^[1]The core interaction logic of this inventive software application is a graphic step-sequencer wherein animated graphical bugs trigger musical notes and/or change their direction of movement when they encounter fat, colored pixels placed in their path by the user. These colored squares define the audiovisual terrain for the four virtual bugs that crawl across the grid surface of the canvas. When one of these bugs encounters a colored square, a diatonic musical note is triggered whose pitch is linked to the square’s color. Each bug represents a different musical instrument and has its own timbre with which it sonifies the squares; thus, one bug produces piano sounds when it collides with the pixels, whereas the other bugs produce percussion, bass guitar, or trumpet sounds.

The user can add, modify, and delete pixels while the bugs are engaged in performing the score. Additional sophistication is possible through the use of certain specially colored pixels, which have the effect of rotating or reversing the bugs that touch them. With these special pixels, the user can cause the bugs to create looping rhythms, phasing polyrhythms, and complex passages which seem to never repeat at all. In this way, the colored squares define a musical score whose results may be generative, unpredictable, highly rhythmic, or some combination of these. And, at the same time, the user of Music Insects is also authoring an image.

Iwai’s Music Insects comes remarkably close to offering a completely balanced solution for authoring image and sound simultaneously. Iwai overcomes many of the legibility problems often associated with symbolic and diagrammatic scores, for example, through the use of his animated bugs, which act as self-revealing and self-explanatory playback heads for the live sound. Because the system’s score elements are elementary pixels as opposed to well-defined symbols, moreover, the fine granularity of Iwai’s audiovisual substance is well suited to the creation of abstract or representational images, and the system’s display screen may be read equally well as a painting or a score.

A playful approach to audiovisual interaction design and musical experience such as that in Music Insects is a hallmark of many Japanese software projects and can also be observed, for example, in software artworks like Haruo Ishii’s Hyperscratch (various versions, 1993–2003) and Masaki Fujihata’s Small Fish (2000).

Bibliography

Brown, Azby. “Portrait of the Artist as a Young Geek.” Wired 5.05 (May 1997); also online at http://www.wired.com/wired/archive/5.05/ff_iwai_pr.html.

“Electroplankton.” On the website Wikipedia: The Free Encyclopedia, retrieved April 15, 2009,http://en.wikipedia.org/wiki/Electroplankton.

Levin, Golan. Painterly Interfaces for Audiovisual Performance. Master’s thesis, Massachusetts Institute of Technology, 2000. Also available online at http://www.flong.com/texts/publications/thesis/.

“Otocky.” On the website Wikipedia: The Free Encyclopedia, retrieved April 15, 2009, http://en.wikipedia.org/wiki/Otocky.

“SimTunes”. On the website Wikipedia: The Free Encyclopedia, retrieved April 15, 2009, http://en.wikipedia.org/wiki/SimTunes.