2 The Interaction and Singularizing of the Senses

Although the significance of such mode-specific qualities cannot be emphasized enough, the particular achievement of our faculty of the senses lies in the linkage and convergence of these supposedly separate domains — it is only their synergetic interaction that has given human beings their evolutionary advantage.

Despite the creation of a means of segregating information on a sense-by-sense basis, evolution did not eliminate the ability to benefit from the advantages of pooling information across sensory modalities. Rather, it created an interesting duality: some parts of the brain became specialized for dealing with information within individual senses, and others for pooling information across senses. ^[3]

Only in the rarest cases are we confronted with sensory stimuli of a single modality, since we perceive our environment with five senses and thus in a multimodal way. In fact, separating our perception into several independent worlds — hearing, seeing, smelling, tasting, feeling — is an enormous task of abstraction that human beings only learn as part of their cultural socialization. In the literature, one sometimes comes across the informative observation that the media and technical apparatuses of the nineteenth century contributed to the singularizing of our senses. For sound, movement, body, and image had an immediate relationship in the history of culture at least until technical possibilities enforced their separation … People stared as if paralyzed at the horns of phonographs. From that time onward, specializing of certain bodily functions outside the body, and hence a specializing of the senses, was necessitated.^[4] Whether or not one chooses to accept this specific assessment, it cannot be denied that technical development and human perception are closely intertwined. Ever since developments in twentieth-century media have made synchronized recording and playback of image and sound possible, our perception and the use and the weighting of our senses has transformed again: the separation of hearing and seeing has been undermined by technology. Whereas the primacy of the eye that dominates in Western culture has come to seem increasingly problematic, the interaction of the senses has been recognized as more important and been discussed more and more frequently. This is another reason why it makes sense to speak of audiovisual perception. Several mechanisms involved in the complex interplay of hearing and seeing will be presented below.