4 The Rules of Space and Time

Multisensory perception is tied to the fundamental question as to which circumstances lead us to perceive qualitatively distinct sensory stimuli as deriving from a common source or as belonging to different objects. Very simple rules have been identified in answer to this question: spatial proximity (the rule of space) and occurrence at the same time (the rule of time) are crucial to the integration of stimuli of different modalities, and even an approximate fulfillment of these conditions is sufficient.^[7] This makes sense, since absolute simultaneity will never be possible, if only because light and sound travel, and are perceived at different speeds. Relative proximity and simultaneity (synchronicity) are thus elementary preconditions for the integration of acoustic and visual information. Thus, it has been demonstrated experimentally that speaking can fall as much as 250 ms behind its visual equivalent — that is, lip movements — before the lack of simultaneity is noticed. But when speaking precedes the lip movements, the different is noticed more quickly. This is analogous to physical reality, since sound always arrives after light. The sometimes surprising effects that result when contradictory sensory stimuli of different modalities are synthesized — for example, the fact that we perceive the voice of a ventriloquist’s dummy as coming from the dummy — are known as crossmodal illusions.