1. Time and space in phoneme perception.

2. Kei Omata & Ken Mogi

Kei Omata (presenting author) Department of Computational Intelligence and Systems Science, Tokyo Institute of Technology, Tokyo, Japan

Ken Mogi (Corresponding Author) Sony Computer Science Laboratories, Tokyo, Japan.

3. Abstract

Phoneme perception is an important step in language cognition. The McGurk effect (McGurk & MacDonald 1976), where a simultaneous presentation of incompatible visual and auditory stimuli leads to a misperception of phonemes, demonstrates that audiovisual integration plays an essential role in phoneme perception. Here, we report a series of psychophysical experiments aimed at elucidating the spatio-temporal constraints involved in the integration process. Using modern technologies, incompatible visual and auditory stimuli were presented with various timing and localization conditions. Presentation of the visual and auditory stimuli in the right egocentric field resulted in a stronger McGurk type effect, possibly related to the left hemisphere dominance of language processing. The effect of the proximity of audio and visual stimuli also exhibited a marked lateral asymmetry. Taking into account the results on the effect of timing in experiments of our own and others (e.g. Munhall et al.1996), we construct a model of the spatio-temporal ﾒstreamliningﾓ involved in the integration process. Our model puts phoneme perception within the context of the more general neural mechanisms of multimodal integration, with a particular emphasis on the relation to the egocentric spatio-temporal perception, such as body image.