5 Perception-Intensifying Effects

Now the question arises as to which mechanisms of multimodal integration can lead to improved perception.^[8] Stein et al. report an experiment in which the sensitivity of individual multimodal neurons was tested.^[9] The reaction to the bimodal stimulus (flash of light plus beep) corresponded approximately to the sum (additive) of reactions to unimodal stimuli. By analogy to the rule of space and the rule of time, a close connection between spatial and temporal coincidence was confirmed on a neuronal basis: the reaction to spatially disparate stimuli was lower than the sum of the two separate stimuli (subadditive), which corresponds to a weakening of the sensation. Stein and Meredith were also able to show particularly strong (superadditive) effects with relatively weak but coincidentally bimodal stimuli.^[10] Thus, reactions triggered by bimodal stimuli are already distinct on the neuronal level from mode-specific components of reaction, which explains in part the perception-intensifying effect of combinations of images and sounds. The phenomena described are probably the neurological basis for Michel Chion’s programmatic observation: We never see the same thing when we also hear; we don‘t hear the same thing when we see as well.^[11]