J Neurophysiol 113: 883 889, 2015.
First published November 12, 2014; doi:10.1152/jn.00284.2014.
What does a neuron learn from multisensory experience?
Jinghong Xu,1* Liping Yu,1* Terrence R. Stanford,2 Benjamin A. Rowland,2 and Barry E. Stein2
1
School of Life Science, East China Normal University, Shanghai, China; and 2Department of Neurobiology and Anatomy,
Wake Forest School of Medicine, Winston-Salem, North Carolina
Submitted 11 April 2014; accepted in final form 11 November 2014
Xu J, Yu L, Stanford TR, Rowland BA, Stein BE. What does
a neuron learn from multisensory experience? J Neurophysiol
113: 883 889, 2015. First published November 12, 2014;
doi:10.1152/jn.00284.2014.The brains ability to integrate infor-
mation from different senses is acquired only after extensive sensory
experience. However, whether early life experience instantiates a
general integrative capacity in multisensory neurons or one limited to
the particular cross-modal stimulus combinations to which one has
been exposed is not known. By selectively restricting either visual-
nonvisual or auditory-nonauditory experience during the first few
months of life, the present study found that trisensory neurons in cat
superior colliculus (as well as their bisensory counterparts) became
adapted to the cross-modal stimulus combinations specific to each
rearing environment. Thus, even at maturity, trisensory neurons did
not integrate all cross-modal stimulus combinations to which they
were capable of responding, but only those that had been linked via
experience to constitute a coherent spatiotemporal event. This selec-
tive maturational process determines which environmental events will
become the most effective targets for superior colliculus-mediated
shifts of attention and orientation.
visual; auditory; somatosensory; superior colliculus; cat
THE BRAIN USES DIFFERENT SENSORY inputs synergistically to
maximize the detection and identification of environmental
events (Stein and Meredith 1993). This fundamental capacity
for multisensory integration is critical for survival, but is
neither inborn nor predetermined and, as shown by studies of
multisensory neurons in the superior colliculus (SC), is ac-
quired only after extensive postnatal experience with cross-
modal cues. Only then can cross-modal inputs be integrated to
enhance the responses of SC neurons and the detection and
orientation behaviors that depend on them (Stein 2012).
However, the nature of the maturational change that takes
place in these neurons is not yet clear. One assumption is that
once a neuron has had sufficient experience to develop the
underlying neural mechanisms required for multisensory inte-
gration, it is able to integrate any of the cross-modal inputs to
which it is responsive (Wallace et al. 2004). In short, its
general integrative capability has matured. Two complemen-
tary observations support the view. First, that SC neurons
trained to integrate a single pair of cross-modal inputs (i.e.,
visual-auditory) are capable of integrating them even when the
physical features of the visual and auditory stimuli are sub-
stantially altered (Xu et al. 2012), and second, that early
deprivation in one sensory modality not only affects multisen-
sory integration involving that modality, but also the integra-
* J. Xu and L. Yu contributed equally to this work.
Address for reprint requests and other correspondence: B. E. Stein, Dept. of
Neurobiology and Anatomy, Wake Forest School of Medicine, Winston-
Salem, North Carolina 27157 (e-mail: bestein@wakehealth.edu).
www.jn.org 0022-3077/15 Copyright 2015 the American Physiological Society
tion taking place among other modalities (Hotting et al. 2004;
Occelli et al. 2012; Wallace et al. 2004).
Nevertheless, explicit proof for this hypothesis has not yet
been provided. The alternative possibility, that neurons do not
develop a general integrative capability, but one specific to
Downloaded from http://jn.physiology.org/ by 10.220.33.5 on March 20, 2017
each cross-modal pairing, cannot yet be excluded. Although
this alternative would involve a more nuanced experiential
process, and greater sophistication in the learning rules oper-
ating at the level of the individual multisensory neuron, it is no
less possible. Distinguishing between these alternatives would
be straightforward if, for example, obtaining the cross-modal
experience with visual and auditory cues necessary for a
trisensory (visual-auditory-somatosensory) SC neuron to inte-
grate visual and auditory inputs could be shown to be sufficient
for it to integrate auditory and somatosensory cues and vice
versa. The present experiments directly examined this issue by
rearing animals in environments that compromised either vi-
sual or auditory experience and then examining the ability of
trisensory (as well as bisensory) SC neurons to integrate the
different cross-modal stimulus pairs to which they were re-
sponsive. The results indicate that SC neurons do not incorpo-
rate a general integrative rule, but instead must learn each
pairwise cross-modal association independently.
MATERIALS AND METHODS
All experimental protocols complied with the National Institutes of
Health Guide for the Care and Use of Laboratory Animals, 8th Ed.
(NRC 2011), and were approved by the Animal Care and Use
Committee of Wake Forest School of Medicine, an Association for the
Assessment and Accreditation of Laboratory Animal Care Interna-
tional-accredited institution.
Experimental Groups
A total of nine cats (six males and three females) were used: three
were reared in standard housing conditions, three were reared in a
dark room from birth, and three were raised from birth in a room with
continuous 80-dB omnidirectional broadband noise provided by
speakers (n 5) located on all sides of their pen. Daily care
observations and routine veterinary procedures for dark-reared ani-
mals were conducted using infrared goggles and an infrared viewing
system that was activated periodically to allow the animals to be
monitored from an adjacent room. In all other relevant respects, the
housing and care provided was the same for all animals.
Efforts were made to preclude any confounding visual and auditory
experiences in the dark-reared and noise-reared groups by anesthetiz-
ing animals in their home cages and maintaining anesthesia through-
out experimentation. Thus, care was taken to minimize the exposure
of animals to restricted modalities. Despite these efforts, however, no
sensory restriction paradigm can be absolute and simultaneously
allow evaluation of those senses. Minimal cross-modal experience can
be acquired by animals during electrophysiological testing, albeit no
883
884 WHAT DOES A NEURON LEARN FROM MULTISENSORY EXPERIENCE?

experimentally-induced effects were noted over the comparatively
short series of recording experiments. In the case of noise-reared
animals, some patterned auditory stimuli were expected to evade
masking by the omnidirectional noise stimulus. Noise-reared and
dark-reared cats began recording sessions at 1214 mo of age, and had
9 12 and 8 15 recording experiments respectively in both SCs.
Implantation Surgery and Electrophysiological Recording
Animals were anesthetized with ketamine hydrochloride (20 mg/kg
im) and acepromazine maleate (0.1 mg/kg im), intubated through the
mouth, and then placed in a stereotaxic apparatus. Isoflurance anes-
thesia (2 4%) was then induced and maintained (1.52%) during
surgery. Expiratory CO2, blood pressure, and heart rate were contin-
uously monitored using a digital vital signs monitor (VetSpecs
VSM7), and body temperature was maintained at 3738C with a
heating pad. A recording chamber that provided access to the SC was
three times the elevation of the action potential amplitude above
background noise. The neural activity was recorded and amplified and
routed to an oscilloscope, audio monitor, and computer for online and
offline analyses. At the end of the recording session, saline (40 50
ml) was given subcutaneously as needed, and anesthetic and paralytic
agents were discontinued. Once stable respiration and locomotion
returned, the animal was returned to its home cage.
Sensory-responsive neurons were identified using a variety of
visual, auditory and somatosensory search stimuli. Visual search
stimuli consisted of moving or flashed bars of light back-projected
from an LC 4445 Philips projector onto a tangent screen placed in
front of the animal. Auditory search stimuli consisted of broadband
(20 20,000 Hz) noise bursts and tones delivered from speakers placed
around the animal and manually produced sounds. Somatosensory
search stimuli consisted of taps, brushes, and manual manipulation of
deep tissue and movement of joints. Whenever a multisensory neuron
was identified, its receptive fields were mapped using conventional

Downloaded from http://jn.physiology.org/ by 10.220.33.5 on March 20, 2017

placed over a craniotomy and attached to the skull with screws and
dental acrylic. Dexamethasone sodium phosphate was administered
prior to surgery (1 mg/kg im, once daily, 1 day prior to surgery; once
daily perisurgery) and after surgery (0.5 mg/kg im, twice daily for 3
days postsurgery; 0.25 mg/kg im, twice daily on postsurgery day 4;
once daily on postsurgery day 5) to reduce the likelihood of brain
edema. After surgery, analgesic (buprenorphine 0.005 0.01 mg/kg
im) was administered twice daily for 3 days. In addition, antibiotic
(ceftriaxone, 20 mg/kg im) was administered twice daily for 7 days.
After a postsurgical recovery period of at least 7 days, weekly
recording began. The animal was anesthetized with ketamine hydro-
chloride (20 mg/kg im) and acepromazine maleate (0.1 mg/kg im),
intubated, and artificially respired. Paralysis was induced with an
injection of pancuronium bromide (0.1 mg/kg iv) to prevent ocular
drift. Anesthesia, paralysis, and hydration were maintained by con-
tinuous intravenous infusion of ketamine hydrochloride (510
mgkg1h1), pancuronium bromide (0.05 mgkg1h1), and 5%
dextrose in sterile saline (2.4 3.6 ml/h). Body temperature was kept
at 37 38C. Respiratory rate and volume were adjusted to keep the
end-tidal CO2 at 4.0%. Expiratory CO2, heart rate, and blood
pressure were monitored continuously, and drug administration ad-
justments were made as necessary to maintain anesthesia.
Conventional methods were used for single-neuron electrophysio-
logical recording as reported in the past (Xu et al. 2012). A glass-
coated tungsten electrode (tip diameter: 13 m, impedance: 13 M
at 1 kHz) was advanced through the multisensory (i.e., deeper) layers
of the SC using a hydraulic microdrive to search for single neurons.
An isolated single neuron was identified using a threshold criterion of
methods (Xu et al. 2012). Visual, auditory and somatosensory recep-
tive fields were transferred to standardized representations of visual-
auditory space and a diagram of the body surface (Stein and Meredith
1993).
To maximize the sample of trisensory neurons, each neuron en-
countered was subjected to an extensive battery of tests with the three
types of stimuli interleaved to determine its modality-convergence
pattern (i.e., trisensory, bisensory, unisensory). Once this pattern was
determined, each multisensory neurons unisensory and multisensory
responses were quantified using randomly interleaved modality-spe-
cific and cross-modal stimulus pairs at intertrial intervals of 57 s.
Generally, this involved interleaving stimuli from each cross-modal
stimulus pair and their component stimuli (3 sequences of 3 condi-
tions). However, in some cases, all combinations involving a visual
stimulus were combined into a single interleaved sequence (5 condi-
tions: visual, auditory, somatosensory, visual-auditory, visual-somato-
sensory) before running the auditory-somatosensory sequence (3 con-
ditions: auditory, somatosensory, auditory-somatosensory). Visual
stimuli (100- to 200-ms duration) were rectangular bars of light (6
2) of varying intensity (1.113.5 cd/m2 against a 0.86 cd/m2 back-
ground) and moved in the optimal direction and speed. The auditory
stimuli consisted of brief (100 200 ms) broadband noise bursts
(20 20,000 Hz) of varying intensity (5570 dB sound pressure level
against an ambient background of 51.452.7 dB sound pressure
level), originating from any of 15 hoop-mounted speakers placed 15
apart and 15 cm from the animals head on a rotating hoop. Somato-
sensory stimuli consisted of vertical displacements of the hair and skin
with a computer-controlled tactile probe. The amplitude and velocity

Table 1. Trisensory neurons: incidence and magnitude of multisensory integration

VA VS AS
Integration Nonintegration Integration Nonintegration Integration Nonintegration

Control
Incidence 84 (47/56) 16 (9/56) 77 (43/56) 23 (13/56) 82 (46/56) 18 (10/56)
ME 104 7 24 7 88 6 24 6 106 7 16 6
Mean ME 91 7 73 6 90 7
Dark-reared
Incidence 17 (8/47) 83 (39/47) 11 (5/47) 89 (42/47) 77 (36/47) 23 (11/47)
ME 87 11 64 90 16 84 104 8 14 9
Mean ME 20 6 17 5 83 8
Noise-reared
Incidence 22 (11/51) 78 (40/51) 75 (38/51) 25 (13/51) 25 (13/51) 75 (38/51)
ME 85 8 11 5 117 9 15 6 94 9 34
Mean ME 27 6 91 9 26 7

Incidence: percentage of neurons showing multisensory enhancement, which is defined as a significant increase in the number of impulses to the
combined cross-modal stimuli compared with the most effective single-modality stimulus (t-test, P 0.05). ME, the magnitude of multisensory
enhancement. ME [(CM SMmax)/SMmax] 100, where CM represents the mean magnitude of the multisensory response and SMmax represents the
magnitude of the response evoked by the more effective modality-specific stimulus. VA, visual-auditory; VS, visual-somatosensory; AS, auditory-somatosensory.

J Neurophysiol doi:10.1152/jn.00284.2014 www.jn.org

 WHAT DOES A NEURON LEARN FROM MULTISENSORY EXPERIENCE? 885

Table 2. Bisensory neurons: incidence and magnitude of multisensory integration

VA VS AS
Integration Nonintegration Integration Nonintegration Integration Nonintegration

Control
Incidence 77 (58/75) 23 (17/75) 71 (29/41) 29 (12/41) 87 (26/30) 13 (4/30)
ME 98 6 11 7 91 9 26 7 90 7 53 4
Mean ME 78 7 72 8 86 8
Dark-reared
Incidence 10 (7/73) 90 (66/73) 14 (4/28) 86 (24/28) 72 (18/25) 28 (7/25)
ME 58 10 73 60 12 1 4 100 12 31 3
Mean ME 12 3 86 81 11
Noise-reared
Incidence 19 (14/74) 81 (60/74) 74 (29/39) 26 (10/39) 22 (5/23) 78 (18/23)
ME 84 9 43 117 11 17 5 76 9 10 6
Mean ME 19 5 91 11 24 8

Downloaded from http://jn.physiology.org/ by 10.220.33.5 on March 20, 2017

of the probe could be varied independently. Stimulus intensities were
adjusted to evoke weak unisensory responses (consisting of few im-
pulses) in each neuron independently to maximize the amplitude of
multisensory enhancement (ME) according to the principle of inverse
effectiveness (Meredith and Stein 1986) in each neuron of each group.
Impulse times (1-ms resolution) were recorded for each trial and
analyzed offline. The response window was defined for each neuron in
the same way by using a geometric algorithm based on the cumulative
impulse count described in an earlier study (Rowland et al. 2007). The
magnitude of each response was identified as the mean number of
impulses occurring in the response window minus the expected
number given the spontaneous firing rate (mean spontaneous firing
rate for each condition was calculated in the 500-ms window preced-
ing the stimulus). The response to the combined cross-modal stimuli
was statistically compared with the response evoked by the more
effective modality-specific stimulus (t-test, P 0.05). Multisensory
enhancement was defined as a significant increase in the number of
impulses to the combined cross-modal stimuli compared with the most
effective single-modality stimulus. The magnitude of this multisensory
response enhancement was evaluated with ME: ME [(CM SMmax)/
SMmax] 100, where CM represents the mean magnitude of the
multisensory response, and SMmax represents the magnitude of the
response evoked by the more effective modality-specific stimulus
(Meredith and Stein 1983).
Data were compared statistically to determine significant differ-
ences. Incidence values were compared across groups using 2 tests;
ME values were compared using t-tests, Mann-Whitney rank sum
tests; and ANOVA was used to evaluate trends where more than three
groups were involved.
RESULTS
Three groups of cats (n 3 in each) were used in the
present experiments: a normal control group, a dark-
reared group reared without light exposure, and a noise-
reared group reared with omnidirectional sound (see MATERIALS
AND METHODS). A total of 562 SC neurons (154 trisensory and
408 bisensory) were recorded in these animals (see Tables 1
and 2), with all experimental groups exhibiting all possible
modality convergence patterns (Fig. 1). However, because
trisensory neurons were expressly sought in each animal and in
each condition by testing each neuron encountered in each
electrode penetration with stimuli in all three modalities indi-
vidually and in combination, their incidence, shown in Table 1,
proved to be higher than previously reported (Kadunce et al.
2001; Meredith and Stein 1986). A neurons integration of
concordant cross-modal signals was indicated by a signifi-
cantly more robust multisensory response than the response to
either component stimulus (Stein et al. 2009). The incidence of
neurons showing elevated multisensory responses for each
stimulus combination, and the mean proportional multisensory
response elevation (or ME, a measure of the effectiveness of
multisensory integration), were calculated for each population
and cross-modal pairing. There were no reliable differences in
the observed pattern of results across animals within experi-
mental groups or across experimental sessions. There were no
significant differences in the visual and auditory stimulus
intensities used for testing neurons ME capabilities in the
three groups (Kruskal Wallis test, auditory stimulus: P 0.25;
visual stimulus: P 0.718).
Trisensory Neurons
Trisensory neurons in the control group displayed an equal
likelihood (P 0.605, 2 test) of integrating each of the different
cross-modal stimulus pairs (Table 1). Furthermore, ME (see
MATERIALS AND METHODS) showed no significant variation across

Control Dark-reared Noise-reared

VAS Visual VAS Visual VAS
17% 20% 13% 16% 14% Visual
(56)
AS (47) (58) AS (51) 28%
(65) (102)
6% (23)
AS 9% 10%
7% (25)
18% Auditory
(30) Auditory VS 8% (28) VS 11% (39)
(31) (63)
13% 8% 7% (25)
(41) 21% 17% Auditory
VS 23% (25) Somato- 21% 13%
(73) (61) (74)
(75) sensory (47)
VA Somato- VA Somato-
VA sensory sensory

Fig. 1. The convergence patterns of superior colliculus (SC) neurons studied in each rearing condition. The neuronal population in control, dark-reared and
noise-reared animals had all possible convergence patterns among the three senses represented in the SC. However, there was a small but statistically significant
decrease in the incidence of visually-responsive neurons in dark-reared animals, and a corresponding change in the incidence of auditory-responsive neurons in
noise-reared animals. VA, visual-auditory, VS, visual-somatosensory; AS, auditory-somatosensory; VAS, visual-auditory-somatosensory.

J Neurophysiol doi:10.1152/jn.00284.2014 www.jn.org

886 WHAT DOES A NEURON LEARN FROM MULTISENSORY EXPERIENCE?

the pairs (P 0.138, one-way analysis of variance, range: than normal (Wallace et al. 2004; Yu et al. 2010), many of
7391%). A typical trisensory neuron is illustrated in Fig. 2A. them integrated auditory-somatosensory cues (77%, 36/47;
Dark-reared group. Although the visual receptive fields of P 0.652) with normal ME levels (P 0.54, t-tests see Table
dark-reared neurons were larger and had poorer spatial fidelity 1 and Fig. 2B). In short, there was no apparent rearing effect on

X impulses / Trial
A 10 230% 250
V A V 8
A Normal 6
** 200
150

(%)
4 sum 100
Visual 150 ms 2 50
0 0
V A VA ME
S 10 250
V S V 8 **187% 200
6 150
sum
4 100
2 50
Auditory 0 0
V S VS ME

Downloaded from http://jn.physiology.org/ by 10.220.33.5 on March 20, 2017

Somatosensory S 10 250
A S A 8 **188% 200
6 sum 150
4 100
2 50
0 0
A SAS ME

A 10 200
B Dark-reared V A V 8
sum
150
6
4 100
Visual 2 50
13%
0 0
S 10 V A VA ME
200
V S V 8
150
6 sum
4 100
2 36% 50
Auditory 0 0
V S VS ME
S 10 200
Somatosensory A S A 8 152%
150
6
sum
** 100
4
2 50
0 0
A S AS ME

A 25 150
C Noise-reared V A V 20
15 100
sum
10 50
Visual 5 19%
0 0
V A VA ME
S 25 150
V S V 20 sum
**104%
15 100
10 50
5
Auditory 0 0
V S VS ME
S 25 150
Somatosensory A S A 20
15 sum 100
10 50
5 17%
0 0
A S AS ME
Fig. 2. The development of multisensory integration capabilities in trisensory SC neurons following different rearing conditions. A: multisensory enhancement in a
normal trisensory neuron was produced by any cross-modal stimulus combination. Left: the sensory convergence pattern. Middle: rasters (ordered bottom to top) illustrate
responses to the 3 modality-specific and 3 cross-modal stimuli. Right: the summary bar graph illustrates the mean responses elicited by each stimulus condition, and the
resultant ME, a measure of multisensory enhancement. Note that each cross-modal stimulus combination produced responses exceeding the predicted sum of the
unisensory responses, a frequent consequence of combining weakly effective stimuli. B: a typical neuron in a dark-reared animal. It was incapable of integrating visual
and nonvisual stimuli, but integrated auditory-somatosensory stimuli to produce the characteristic multisensory response. C: a typical neuron from a noise-reared animal.
Despite failure to integrate auditory and nonauditory information, visual-somatosensory integration is intact. **P 0.001.

J Neurophysiol doi:10.1152/jn.00284.2014 www.jn.org

 WHAT DOES A NEURON LEARN FROM MULTISENSORY EXPERIENCE? 887

their ability to integrate cues with which they had extensive
experience.
However, significantly fewer of them were capable of visual-
nonvisual integration than normal (P 0.001 in both cases).
Visual-auditory integration took place in only 17% (8/47) of
them, and visual-somatosensory integration in only 11% (5/47)
(Table 1). Furthermore, their respective MEs (20 and 17%)
were also significantly lower than normal (P 0.001 in both
cases) and lower than their ME for auditory-somatosensory
integration (P 0.001 in both cases). The inability to integrate
visual-nonvisual cues occurred despite the fact that the unisen-
sory comparator response magnitudes used to determine mul-
tisensory integration were statistically indistinguishable from
those used in auditory-somatosensory integration (P 0.169)
and from those used in the control cohort (P 0.073). Thus the
absence of visual experience selectively compromised the
mean ME 91%). However, they were far less likely than
normal to integrate auditory-nonauditory stimulus combina-
tions (see Table 1 and Fig. 2C), despite the fact that the
noise-rearing paradigm provides an imperfect masking of au-
ditory stimuli in the environment.
Only 22% (11/51) of trisensory neurons became capable of
integrating a visual-auditory stimulus combination, and only
25% (13/51) became capable of integrating an auditory-so-
matosensory stimulus combination: much lower (P 0.001 in
both cases) incidences than in the control cohort, and lower
(P 0.001 in both cases) than their own probability of
visual-somatosensory integration. Similarly, ME levels for
each auditory-nonauditory combination were significantly
(P 0.001 in both cases) lower (visual-auditory 27%, and
auditory-somatosensory 26%) than in the control cohort and
significantly (P 0.001 in both cases) lower than for their own

Downloaded from http://jn.physiology.org/ by 10.220.33.5 on March 20, 2017

development of visual-nonvisual integration in trisensory neu-
rons (see Fig. 3).
Noise-reared group. Functional homologies with dark-
reared neurons were apparent. These neurons had larger than
normal auditory receptive fields (Efrati and Gutfreund 2011),
evidencing poor overlap with their nonauditory counterparts
receptive fields. Similarly, extensive visual-somatosensory ex-
perience in the rearing environment was reflected in a normal
(P 0.961) proportion (75%, 38/51) of neurons that had
developed the ability to integrate this cross-modal stimulus
combination. They also did so at normal levels (P 0.207;
visual-somatosensory pairings. The incidence and mean ME of
auditory-somatosensory integration were also lower than those
for auditory-somatosensory in the dark-reared group (inci-
dence: P 0.001, ME: P 0.001).
As in the case of dark-reared neurons, these differences
could not be attributed to differences in the unisensory com-
parator responses (P 0.882). Thus this rearing condition
selectively compromised the development of trisensory neu-
rons ability to synthesize auditory-nonauditory stimuli (see
Fig. 3). Interestingly, the incidence and effectiveness of visual-
auditory integration were similar in both noise-reared and

A All Neurons B All Neurons C Integrating Neurons Only

Normal
% Neurons with Multi. Integ.

VA VS AS Dark-reared
100 Normal
100 Noise-reared
Probability A Neuron

Dark
80 Noise 100
Would Integrate

80
80
ME(%)

60
60 60
40 40 40
20 20 20
0 0 0
VAVSAS VAVSAS VAVSAS With cross-modal Without cross-modal VS VA AS
Normal Dark-reared Noise-reared experience experience

D Normal E Dark-reared F Noise-reared

300 300 300
VA
200 VS 200 200
AS
ME (%)
ME (%)

ME (%)

100 100 100

0 0 0

0 2 4 6 8 10 12 14 16 18 0 2 4 6 8 10 12 14 16 18 0 2 4 6 8 10 12 14 16 18
Best Unisensory Response Best Unisensory Response Best Unisensory Response
Mean Impulses / Trial Mean Impulses / Trial Mean Impulses / Trial
Fig. 3. Summary of multisensory integration for trisensory neurons in each rearing condition. A: as shown here, similar percentages of neurons in normally-reared
animals are able to integrate the 3 different combinations of cross-modal sensory inputs. However, integration was compromised in the cross-modal pairs
involving the deprived modality. B: the averaged ME was high (indistinguishable from normal), for those cross-modal combinations with which neurons had
experience, and very low for those with which they did not. C: in normally-reared animals, neurons capable of integrating one cross-modal stimulus combination
(integrating neurons) also had a high probability of integrating others, but this was not true in dark-reared and noise-reared animals. Their integration
capabilities were largely restricted to those combinations with which they had experience. DF: MEs are plotted for each neuron in each group. Note that neurons
with ME values greater than 100% reflect a superadditive comutation. The stars in D, E, and F represent the example neurons shown in Fig. 2.

J Neurophysiol doi:10.1152/jn.00284.2014 www.jn.org

888 WHAT DOES A NEURON LEARN FROM MULTISENSORY EXPERIENCE?

dark-reared populations (incidence: P 0.754; ME: P 0.344),
presumably because each rearing condition compromised one of
the modalities involved in that same cross-modal pair.
Bisensory Neurons
The population of bisensory neurons showed developmental
profiles that paralleled those of trisensory neurons (Table 2 and
Fig. 4). Dark-rearing yielded a significantly lower (P 0.001
in both cases) incidence of visual-nonvisual neurons capable of
multisensory integration (visual-auditory: 10%, 7/73; visual-
somatosensory: 14%, 4/28). However, it had no significant
impact on the incidence (72%, 18/25, P 0.31) or the
effectiveness (81%, P 0.722) of multisensory integration
in auditory-somatosensory neurons. Similarly, noise-rearing
yielded a significantly lower (P 0.001 in both cases) inci-
through which it is then able to synthesize all combinations of
senses to which it is responsive. The independence with which
different cross-modal associations are learned reveals a previ-
ously unappreciated complexity in the multisensory SC circuit,
one that affords it the capacity to deal very differently with
different cross-modal stimulus combinations, even in the same
neurons. The biological basis of this is not yet understood, but
its key feature is a computational segregation of sensory inputs
by their pairwise affiliation. This segregation may be accom-
plished in a variety of ways at the circuit (Cuppini et al. 2012;
Fetsch et al. 2013) or single-neuron level (Hausser and Mel
2003), or some combination of factors involving both levels.
How this developmental event occurs, whether via Hebbian,
homeostatic, spike timing plasticity, or some other mechanism
requires further research.

Downloaded from http://jn.physiology.org/ by 10.220.33.5 on March 20, 2017

dence of auditory-nonauditory neurons capable of multisensory
integration (auditory-visual: 19%, 14/74; auditory-somatosen-
sory: 22%, 5/23). However, it had little impact on the incidence
(74%, 29/39, P 0.91) or effectiveness (91%, P 0.18) of
visual-somatosensory integration. As in the trisensory neurons,
there were no significant differences (P 0.05 in each case) in
the vigor of comparator unisensory responses among these
different cross-modal combinations.
DISCUSSION
The present results reveal that the SC multisensory circuit
learns to integrate each combination of the senses individually
and reflects this in the behavior of its individual neurons. Thus,
although some intuitions and observations may have led to
predictions to the contrary because, for example, of the signif-
icance of vision for the functional development of other senses
(King et al. 1996; Putzar et al. 2007; Wallace et al. 2004; Xu
et al. 2012), a neuron does not develop a general operation
These observations, coupled with prior studies (Xu et al.
2012; Yu et al. 2010), suggest that the development of these
combinatorial operations is based on cross-modal signal con-
cordance in space and time, not the specific features of the
modality-specific stimuli being combined (Xu et al. 2012).
Furthermore, they can develop even if their associated unisen-
sory response properties are immature. Mature unisensory
information processing does not automatically confer the ca-
pacity to integrate across modalities, and mature multisensory
processing does not require unisensory maturity. Thus unisen-
sory and multisensory development are not inextricably linked.
Indeed, sensory deprivation will significantly impact unisen-
sory development in both the visual and auditory modalities
(Chang and Merzenich 2003; Efrati and Gutfreund 2011;
Rauschecker and Korte 1993; Wiesel and Hubel 1965). How-
ever, as shown here, it will not compromise the development of
all multisensory integration capabilities, only those involving
the restricted modality. Restricting visual experience had no

A B
% Neurons with Multi. Integ.

100 VA VS AS
100 Normal
Dark
80 Noise
80
ME (%)

60
60
40 40
20 20
0 0
VA VSAS VA VSAS VA VSAS With cross-modal Without cross-modal
Normal Dark-reared Noise-reared experience experience

C Normal D Dark-reared E Noise-reared

300 300 300
VA
200 VS 200 200
AS
ME (%)

ME (%)
ME (%)

100 100 100

0 0 0

0 5 10 15 20 25 0 5 10 15 20 25 0 5 10 15 20 25
Best Unisensory Response Best Unisensory Response Best Unisensory Response
Mean Impulses / Trial Mean Impulses / Trial Mean Impulses / Trial
Fig. 4. Summary of multisensory integration for bisensory neurons in each rearing condition. Conventions are the same as in Fig. 3.

J Neurophysiol doi:10.1152/jn.00284.2014 www.jn.org

 WHAT DOES A NEURON LEARN FROM MULTISENSORY EXPERIENCE?
greater effect in this regard than did restricting auditory expe-
rience, despite the fact that the nature of this restriction was
quite different. In the former case, visual input was eliminated,
whereas in the latter case auditory experience was common due
to the constant noise, but most transient auditory events were
masked. The equivalence of the rearing conditions in this
regard is, perhaps, surprising given the common view of the SC
as a visually dominant structure.
Furthermore, their equal effectiveness in both dark-reared
and noise-reared animals is interesting, given that the cross-
modal stimuli that they were exposed to during early life did
not produce a greater than normal incidence or magnitude of
multisensory integration. This suggests that, beyond a certain
point, more experience does not have an impact on this pro-
cess. Thus multisensory integration appears to be optimized by
factors other than total experience; most likely, it is calibrated
by the informational benefits conferred by the joint consider-
ation of two conditionally independent sensory channels (Pat-
ton and Anastasio 2003). Thus, while theoretically the system
could generate more and/or larger multisensory responses, it
does not do so, presumably because such responses would
confer increases in salience not warranted by the information
gained.
In summary, it appears that to develop the ability to integrate
signals from different senses, neurons (circuits) must have
experience with them in combination. Whether this merely
reflects that some degree of observed covariance is necessary
to form the requisite cross-modal associations, or reflects the
direct encoding of the probabilities of association, is currently
unknown. However, by being specific to cross-modal pairings,
the developmental algorithm selectively enhances signals that
are most likely to be linked to a common event. It thereby
configures the system to be of particular benefit in complex
environments in which the selection of targets for overt behav-
ior is complicated by the many signals from each modality
(Pluta et al. 2011).
ACKNOWLEDGMENTS
We thank Nancy London for technical assistance.
GRANTS
This research was supported by National Institutes of Health Grants
EY-016716 and NS-036916 and grants from the Wallace Foundation, the
National Natural Science Foundation of China (Grants 31400944 and
31300910), and the Shanghai Natural Science Foundation (Grant
14ZR1411500).
DISCLOSURES
No conflicts of interest, financial or otherwise, are declared by the author(s).
AUTHOR CONTRIBUTIONS
Author contributions: J.X., L.Y., and B.E.S. conception and design of
research; J.X. and L.Y. performed experiments; J.X. and L.Y. analyzed data;
J.X. and L.Y. interpreted results of experiments; J.X. and L.Y. prepared
J Neurophysiol doi:10.1152/jn.00284.2014 www.jn.org
889
figures; J.X. and L.Y. drafted manuscript; J.X., L.Y., T.R.S., B.A.R., and
B.E.S. edited and revised manuscript; J.X., L.Y., T.R.S., B.A.R., and B.E.S.
approved final version of manuscript.
REFERENCES
Chang EF, Merzenich MM. Environmental noise retards auditory cortical
development. Science 300: 498 502, 2003.
Cuppini C, Magosso E, Rowland B, Stein B, Ursino M. Hebbian mecha-
nisms help explain development of multisensory integration in the superior
colliculus: a neural network model. Biol Cybern 106: 691713, 2012.
Efrati A, Gutfreund Y. Early life exposure to noise alters the representation
of auditory localization cues in the auditory space map of the barn owl. J
Neurophysiol 105: 25222535, 2011.
Fetsch CR, DeAngelis GC, Angelaki DE. Bridging the gap between theories
of sensory cue integration and the physiology of multisensory neurons. Nat
Rev Neurosci 14: 429 442, 2013.
Hausser M, Mel B. Dendrites: bug or feature? Curr Opin Neurobiol 13:
372383, 2003.
Downloaded from http://jn.physiology.org/ by 10.220.33.5 on March 20, 2017
Hotting K, Rosler F, Roder B. Altered auditory-tactile interactions in con-
genitally blind humans: an event-related potential study. Exp Brain Res 159:
370 381, 2004.
Kadunce DC, Vaughan JW, Wallace MT, Stein BE. The influence of visual
and auditory receptive field organization on multisensory integration in the
superior colliculus. Exp Brain Res 139: 303310, 2001.
King AJ, Schnupp JW, Carlile S, Smith AL, Thompson ID. The develop-
ment of topographically-aligned maps of visual and auditory space in the
superior colliculus. Prog Brain Res 112: 335350, 1996.
Meredith MA, Stein BE. Interactions among converging sensory inputs in the
superior colliculus. Science 221: 389 391, 1983.
Meredith MA, Stein BE. Visual, auditory, and somatosensory convergence
on cells in superior colliculus results in multisensory integration. J Neuro-
physiol 56: 640 662, 1986.
Occelli V, Bruns P, Zampini M, Roder B. Audiotactile integration is reduced
in congenital blindness in a spatial ventriloquism task. Neuropsychologia
50: 36 43, 2012.
Patton PE, Anastasio TJ. Modeling cross-modal enhancement and modality-
specific suppression in multisensory neurons. Neural Comput 15: 783 810,
2003.
Pluta SR, Rowland BA, Stanford TR, Stein BE. Alterations to multisensory
and unisensory integration by stimulus competition. J Neurophysiol 106:
30913101, 2011.
Putzar L, Goerendt I, Lange K, Rosler F, Roder B. Early visual deprivation
impairs multisensory interactions in humans. Nat Neurosci 10: 12431245,
2007.
Rauschecker JP, Korte M. Auditory compensation for early blindness in cat
cerebral cortex. J Neurosci 13: 4538 4548, 1993.
Rowland BA, Quessy S, Stanford TR, Stein BE. Multisensory integration
shortens physiological response latencies. J Neurosci 27: 5879 5884, 2007.
Stein BE. The New Handbook of Multisensory Processing. Cambridge, MA:
MIT, 2012.
Stein BE, Meredith MA. The Merging of the Senses. Cambridge, MA: MIT,
1993.
Stein BE, Stanford TR, Ramachandran R, Perrault TJ Jr, Rowland BA.
Challenges in quantifying multisensory integration: alternative criteria,
models, and inverse effectiveness. Exp Brain Res 198: 113126, 2009.
Wallace MT, Perrault TJ Jr, Hairston WD, Stein BE. Visual experience is
necessary for the development of multisensory integration. J Neurosci 24:
9580 9584, 2004.
Wiesel TN, Hubel DH. Extent of recovery from the effects of visual depri-
vation in kittens. J Neurophysiol 28: 1060 1072, 1965.
Xu J, Yu L, Rowland BA, Stanford TR, Stein BE. Incorporating cross-
modal statistics in the development and maintenance of multisensory inte-
gration. J Neurosci 32: 22872298, 2012.
Yu L, Rowland BA, Stein BE. Initiating the development of multisensory
integration by manipulating sensory experience. J Neurosci 30: 4904 4913,
2010.