8 Early Concepts of Pattern Playback

Attempts to use visual information in order to control sounds had already begun in the 1940s. At that time, the principle of the sonogram was reversed; instead of recording audio frequencies, visual information was interpreted and sonified as frequencies. This method, termed pattern playback in North American linguistic studies, was further developed with the help of computer technology.^[12] The UPIC developed by composer Iannis Xenakis in 1977, was the first real-time system that directly sonified visual forms. In this system, figures are drawn on a graphics pad and their shapes then prescribe the pitch, while their positioning determines the tone sequence or tone variation.^[13] Likewise in the 1970s, the diffusion of video technology led to the development of various systems that used live video images as the input for the generation of sound, for example Erkki Kurenniemi’s Dimi-O systems^[14] and the Cloud Music project created by Robert Watts, David Behrman, and Bob Diamond (1974–1979), in which a video camera recorded cloud movements, and an analysis of the brightness at six points of the image was used to manipulate sound canals.^[15]