12 New Experiments in Interactive Sound Visualization

Over the last ten years, the visualization of sound produced by recipients, which had already been experimented with in the 1950s and 1960s, has also been further developed. Thus, Levin and Lieberman added sound components to installations based on real-time analysis and projection of shadows. In their installation re:mark, a voice-recognition system attempts to transform visitors’ speech into writing which then—taking its cue from the visitors’ shadows—moves across a screen. The similarly constructed installation messa di voce (Voice Placement), on the other hand, converts sound into abstract shapes.^[23]

In 2002, the two artists together with the Ars Electronica Futurelab developed a completely new approach in their installation The Hidden Worlds of Noise and Voice. Spoken exchanges are made spatially visible: the voices of or noises made by different users sitting at a round table are converted into virtual sound sculptures by means of 3D technology. The forms that emerge can be observed, on one side, through special 3D spectacles, while on the other they are projected as shadows onto the table, so that observers standing next to the users can also follow the visualized process of communication.

Another type of translation of sound and image is produced when the sounds are interpreted at a symbolic level rather than visualized. For example, in Vincent Elka’s installation Shout, the projection of a woman’s face reacts to the acoustic input produced by the visitors. The system attempts to read emotions from their voices and induces the woman to react to them both through her facial mimics and her language.^[24] Here, Elka is referring back to concepts of a linguistic communication between humans and technology that are relevant in AI (artificial intelligence) research, but because they are based on symbolic systems cannot be considered sound-image transformations in the narrow sense. As has been shown, these transformations are mostly dedicated to abstract, and also often associative relations between sounds and colors and forms. As interactive art projects, they also invite the visitor to actively explore them.