Please cite this article in press as: Woolley SC, Kao MH.

Variability in action: Contributions of a songbird cortical-basal ganglia circuit to vocal motor

learning and control. Neuroscience (2014), http://dx.doi.org/10.1016/j.neuroscience.2014.10.010

Neuroscience xxx (2014) xxxxxx

REVIEW
VARIABILITY IN ACTION: CONTRIBUTIONS OF A SONGBIRD
CORTICAL-BASAL GANGLIA CIRCUIT TO VOCAL MOTOR LEARNING
AND CONTROL
S. C. WOOLLEY a* AND M. H. KAO b

Contents
Introduction
Anatomy
Trial-and-error: the role of variability in song
learning and plasticity
Locus for the active generation of variability
Conclusion
References

Department of Biology, McGill University, 1205 Avenue

Dr. Peneld, Montreal, QC H3A 1B1, Canada
b
Department of Physiology, University of California,
San Francisco, 675 Nelson Rising Lane, Room 518, San Francisco,
CA 94158, USA

AbstractMany motor behaviors, from walking to speaking,

are acquired through experience, in particular, through trialand-error learning. The acquisition and maintenance of such
motor behaviors in a wide range of species, including
humans, appear to depend on cortical-basal ganglia circuits. In this review, we discuss recent studies in songbirds
that have been pivotal in informing our current understanding of motor learning and cortical-basal ganglia function.
Songbirds are important ethological model systems for the
study of motor learning because young songbirds naturally
develop and rene their songs through trial-and-error learning. In addition, reinforcement mechanisms are hypothesized to be important for the maintenance and plasticity of
structured adult song. Computational and experimental
studies highlight the importance of vocal motor variability
as the substrate upon which reinforcement mechanisms
could operate to shape developing song and to maintain
adult song. Recent studies in songbirds indicate that this
vocal motor variability is actively generated and modulated
by a highly specialized cortical-basal ganglia circuit evolved
for a single behavior, song. We argue that these and other
recent ndings illustrate how the tight association between
a specialized neural circuit and a natural behavior make
songbirds a unique and powerful model in which to investigate the neural substrates of motor learning and plasticity.
This article is part of a Special Issue entitled: Choice of model
organisms for Neuroscience Research. 2014 Published by
Elsevier Ltd. on behalf of IBRO.

00
00
00
00
00
00

INTRODUCTION
Cortical-basal ganglia circuits are highly conserved
across vertebrates, including humans, and are critical
for learning and performing wide-ranging motor
behaviors (Graybiel, 2005, 2008; Grillner et al., 2005).
Of particular interest is the role of the basal ganglia in
the acquisition and performance of motor behaviors that
are not innate, but also are not explicitly taught. Rather,
these behaviors are thought to follow a reinforcement
learning model whereby the results of actions are evaluated in a trial-and-error fashion, with reinforcement provided to those motor patterns that result in favorable
outcomes and suppression of actions that lead to undesirable outcomes (Sutton and Barto, 1998; Ishii et al., 2002).
There are three components of reinforcement learning
models that researchers have aimed to match with neural
substrates: the brain must (1) generate variable actions
(2) evaluate those actions and (3) reinforce or suppress
future actions accordingly. There has been considerable
and groundbreaking work, especially in primates,
investigating the neural mechanisms by which actions
are evaluated. Neurons in the midbrain ventral
tegmental area and substantia nigra pars compacta
display reward sensitivity that can provide an error or
evaluation signal during learning (Schultz et al., 2003;
Schultz, 2013). In addition, studies in mammals and other
species have examined how cortical-basal ganglia circuits
bias motor output toward a desired outcome. Indeed,
given the motor decits that arise from diseases of the
basal ganglia, including Parkinsons and Huntingtons diseases, the role of the basal ganglia in movement is clearly
critical (DeLong, 1971; Delong and Wichmann, 2007;
Wichmann et al., 2011). Together, these data indicate

Key words: songbird, basal ganglia, reinforcement learning,

variability.

*Corresponding author. Tel: +1-514-398-2324.

E-mail address: sarah.woolley@mcgill.ca (S. C. Woolley).
Abbreviations: AFP, anterior forebrain pathway; DLM, dorsolateral
anterior nucleus of the thalamus; GPe, external globus pallidus; GPi,
internal globus pallidus; HVCX, HVC, cells that project to Area X;
LMAN, magnocellular nucleus of the anterior nidopallium; RA, robust
nucleus of the arcopallium; SNs, spiny neurons.
http://dx.doi.org/10.1016/j.neuroscience.2014.10.010
0306-4522/ 2014 Published by Elsevier Ltd. on behalf of IBRO.
1

S. C. Woolley, M. H. Kao / Neuroscience xxx (2014) xxxxxx

that signals arising in cortical-basal ganglia circuits can

incorporate information about reward or context to bias
behavior on a moment-by-moment basis (Hikosaka
et al., 1989; Kawagoe et al., 1998; Handel and
Glimcher, 2000; Gdowski et al., 2001; Lauwereyns
et al., 2002; Brasted and Wise, 2004; Kao et al., 2005;
Pasupathy and Miller, 2005; Williams and Eskandar,
2006; Andalman and Fee, 2009). Thus, many aspects
of reinforcement learning appear critically dependent on
the basal ganglia (Graybiel et al., 1994; Doya and
Sejnowski, 1998, 2000; Sutton and Barto, 1998; Bar
Gad and Bergman, 2001; Doya, 2002).
Much less is known, however, about how and where
motor variability, an essential component of trial-anderror learning, is generated. Reinforcement models
propose the existence of motor exploration to nd
actions that result in favorable outcomes and
exploitation to eventually perform just those favorable
actions. Moreover, several recent studies have shown
that individual dierences in the degree of motor
variability can predict or constrain the ability to learn or
alter motor skills (Sober and Brainard, 2012; Wu et al.,
2014). While a handful of studies have described dierent
patterns of basal ganglia activity between behavioral
exploration and exploitation states in mammals
(Graybiel, 2005; Barnes et al., 2005; Sheth et al., 2011),
in many respects, the study of songbirds has been crucial
to our current understanding of the nature of trial-by-trial
variability, how it is generated in cortical-basal ganglia circuits and its role in behavioral learning and plasticity
(Schar and Nottebohm, 1991; Kao et al., 2005;
Olveczky et al., 2005; Kao and Brainard, 2006; Tumer
and Brainard, 2007; Andalman and Fee, 2009; Mooney,
2009; Stepanek and Doupe, 2010; Fee and Goldberg,
2011; Goldberg and Fee, 2011; Kojima and Doupe,
2011; Warren et al., 2011; Sober and Brainard, 2012).
In this review, we will discuss a series of studies that
have investigated the emergence of neural variability in
an avian cortical-basal ganglia circuit required for motor
learning and plasticity. These studies take advantage of
the highly stereotyped nature of adult birdsong, where
the sequence and structure of song elements are welllearned and practiced and yet maintain some variability

from trial-to-trial. This variability can subserve

reinforcement learning in adult birds (Tumer and
Brainard, 2007), and, as we will discuss, is not a default
state of the nervous system but rather is actively
generated.

ANATOMY
In songbirds, there are two pathways important for song
learning and song production (Nottebohm et al., 1976;
Bottjer et al., 1984; Nordeen and Nordeen, 1997;
Brainard and Doupe, 2002). In one, known as the motor
pathway, the cortical nucleus HVC (proper name)
receives input from cortical auditory areas and projects
to the song motor control nucleus (robust nucleus of the
arcopallium; RA), which innervates motor and respiratory
neurons used for singing (Fig. 1B). A second pathway, the
anterior forebrain pathway (AFP) also receives input
from HVC. Specically, a subset of HVC neurons project
to the basal ganglia nucleus Area X, which in turn projects
to the cortical area the lateral magnocellular nucleus of
the anterior nidopallium (LMAN) via the medial portion
of the dorsolateral anterior nucleus of the thalamus
(DLM). LMAN provides the output of the AFP to the motor
pathway and also sends a recurrent projection back to
Area X. Our focus for this review is this second pathway,
the AFP (Fig. 1B).
The AFP shares substantial similarity with corticalbasal ganglia circuits in mammals (Fig. 1A, B). HVC and
LMAN are pallial in nature and have been likened to
layers 2/3 of the mammalian motor cortex (Karten,
1969, 1991; Jarvis et al., 2005; Reiner et al., 2005;
Dugas-Ford et al., 2012). Area X is a large nucleus within
the basal ganglia proper that has evolved for a single,
ethologically relevant function-song. Like mammalian
basal ganglia, the nucleus Area X receives both cortical
input and midbrain dopaminergic input, and projects to
the thalamus (Lewis et al., 1981; Bottjer, 1993; Soha
et al., 1996; Luo and Perkel, 1999; Farries and Perkel,
2002; Carrillo and Doupe, 2004; Reiner et al., 2004;
Farries et al., 2005; Person et al., 2008). Additionally,
as in mammals, Area X can inuence its dopaminergic
inputs, although only indirectly via the ventral pallidum

Fig. 1. Simplied diagram highlighting the similarity in structure and connectivity between mammalian cortical-basal ganglia circuits (A) and the
song system in songbirds (B). In both, cortical/pallial premotor structures (blue) provide input to cells and nuclei in the basal ganglia (green). The
basal ganglia receives dopaminergic input from the substantia nigra pars compacta and the ventral tegmental area (red), and sends projections to
the thalamus (purple), which projects back to premotor cortical regions. These premotor regions also project to motor cortical structures which have
descending connections that can ultimately alter motor output.
Please cite this article in press as: Woolley SC, Kao MH. Variability in action: Contributions of a songbird cortical-basal ganglia circuit to vocal motor
learning and control. Neuroscience (2014), http://dx.doi.org/10.1016/j.neuroscience.2014.10.010

S. C. Woolley, M. H. Kao / Neuroscience xxx (2014) xxxxxx

(Gale et al., 2008). While the striatum and pallidum are

distinct structures in other parts of the avian basal
ganglia, Area X comprises intermixed cell types similar
to both striatal and pallidal cell types in mammals. In particular, Area X contains inhibitory spiny neurons (SNs),
cholinergic tonically active cells (TANs), GABAergic and
fast-spiking interneurons (FSIs) and tonically active,
high-ring GABAergic neurons that exhibit activity similar
to cells in the internal (GPi) and external (GPe) globus
pallidus (Farries and Perkel, 2002; Carrillo and Doupe,
2004; Reiner et al., 2004; Goldberg and Fee, 2010;
Goldberg et al., 2010). Like their counterparts in mammalian
basal ganglia, both SNs and GPi neurons in Area X have
been shown to be dopamine sensitive, although the dopamine sensitivity of the other cell types remains to be
determined (Ding and Perkel, 2002; Leblois et al., 2010).
As we will discuss in more detail below, lesions of the
AFP aect the learning and plasticity of song without
disrupting other motor behaviors. Thus, the AFP is a
highly specialized cortical-basal ganglia circuit with a
specic function. This tight association with a single
behavior enables direct comparisons between neural
activity and behavior in order to elucidate the role of
single neurons in the production and modulation of a
learned motor output, and provides the opportunity to
manipulate individual pieces of the circuit to determine
their behavioral relevance (reviewed in Doupe et al.,
2005).

TRIAL-AND-ERROR: THE ROLE OF

VARIABILITY IN SONG LEARNING AND
PLASTICITY
Variability in behavior has often been considered an
unwanted feature or default state of the system that can
be overcome with sucient practice (Osborne et al.,
2005; Churchland et al., 2006a,b). On the other hand, theoretical work posits the importance of variability in reinforcement learning (Sutton and Barto, 1998). During
reinforcement learning, the results of a number of actions
are evaluated in a trial-and-error fashion, with dierential
reinforcement or suppression occurring depending on
the outcome. Such a model is dependent on a balance
between exploration of motor space and exploitation
of motor commands that produce favorable actions.
Moreover, in order to nd an optimal solution, it is often
necessary for an organism to continue to produce a range
of actions, that is, to maintain variability, even after it has
successfully performed an action or reached a behavioral
target. Thus, in reinforcement learning models, the active
generation of variability is critical to determining the maximally protable response.
Studies in mammals indicate that as animals move
from learning to executing a particular task, there are
shifts in the activity of populations of striatal (Barnes
et al., 2005) and pallidal (Sheth et al., 2011) neurons. In
particular, there appears to be a neural signature that corresponds to motor exploration versus motor exploitation.
These neural signals are dynamic, for example disappearing and reappearing as the task is extinguished and
then relearned (Barnes et al., 2005). However, identifying

how particular neurons contribute, on a trial-by-trial basis,

to the active generation of variability has proven to be
challenging. Indeed, both the ubiquity of the basal ganglia
in learning and production of all motor actions, and the
lack of independent association between identiable cortical-basal ganglia loops and specic behaviors, make it
dicult to distinguish between variability as a default state
versus variability that is actively generated.
In songbirds, the relationship between neural
variability and behavioral variability is better established.
Young songbirds memorize the song of their tutor early
in development and then undergo a period of
sensorimotor learning that involves vocal exploration
coupled with auditory feedback to match their immature
vocalizations to the memorized template song (Konishi,
1965; Marler, 1970; reviewed in Brainard and Doupe,
2002). During this time, song progressively improves, until
ultimately the sequence and structure of syllables are
more stereotyped and show impressive similarity to the
tutor song. As adults, zebra nches produce a crystallized song that is well-matched to that of the tutor and
is similar in sequence and structure from one rendition
to the next (Konishi, 1965; Immelman, 1969; Marler,
1970). Interestingly, developing birds are capable of producing stereotyped, adult-like versions of their song
depending on social context (Kojima and Doupe, 2011),
suggesting that much of the song variation, even in younger birds, is actively generated vocal exploration.
Such vocal motor exploration is dependent on the
AFP. Lesions or inactivation of LMAN in juvenile birds
result in stereotyped but impoverished song. In
particular, while syllable sequences and syllable
structure are highly consistent from rendition to rendition
after lesions of LMAN, the song itself is a poor match to
the song of the tutor (Bottjer et al., 1984; Schar and
Nottebohm, 1991; Olveczky et al., 2005; Goldberg and
Fee, 2011). These data highlight the importance of the
AFP to learning in juveniles and perhaps its contribution
to vocal exploration in particular.
While song, especially syllable sequence, is generally
more stereotyped in adults than juveniles, there continues
to be subtle variation in syllable structure. In particular,
song variability is naturally modulated by social context
(Fig. 2AC). Song structure can rapidly shift, from a
variable and more plastic song when a male sings alone
(undirected) to a stereotyped and less plastic courtship
song (directed song) performed for and preferred by
females (Kao and Brainard, 2006; Woolley and Doupe,
2008; Hampton et al., 2009; Sakata and Brainard,
2009). In adults, stimulation of LMAN can drive variability
in syllable features, such as pitch (Kao et al., 2005), while
lesioning or inactivating LMAN reduces variability in syllable structure during undirected singing (Olveczky et al.,
2005; Kao and Brainard, 2006; Hampton et al., 2009;
Stepanek and Doupe, 2010). Moreover, the activity of single neurons in LMAN during directed and undirected singing is modulated by social context (Fig. 2DF; Hessler
and Doupe, 1999; Kao et al., 2008). Relative to activity
during directed singing, individual neurons show higher
ring rates, more burst ring and less reliable ring patterns during undirected singing. While the activity from

Please cite this article in press as: Woolley SC, Kao MH. Variability in action: Contributions of a songbird cortical-basal ganglia circuit to vocal motor
learning and control. Neuroscience (2014), http://dx.doi.org/10.1016/j.neuroscience.2014.10.010

S. C. Woolley, M. H. Kao / Neuroscience xxx (2014) xxxxxx

Fig. 2. Syllable structure and neural activity in LMAN are more variable during undirected singing. (A) Spectrogram of a single zebra nch syllable,
dashed lines indicate the time window where the fundamental frequency was measured. (B, C) Histograms of the fundamental frequency during
undirected (B) and directed (C) singing. These data illustrate the greater cross-rendition variability in syllable pitch during undirected singing. Activity
in LMAN is also more variable during undirected singing. (D) Spectrogram of a song motif. Raster plots of the activity of a single neuron in LMAN
during directed (E) and undirected (F) motif renditions. (F) While there are signicant dierences in the ring rate, variability and amount of bursting
between the two social conditions (compare E and F), the average song-related pattern of activity is highly similar between directed and undirected
singing (Data not shown). Modied from Kao and Brainard (2006) and Kao et al. (2008).

trial-to-trial shows dramatic social modulation, the average pattern across trials is highly similar between the
two social contexts, implying that activity in LMAN provides the motor pathway with variation around a patterned
signal.
Together, these data indicate that, in both juveniles
and adults, behavioral variability is not a default state,
but actively requires cortical-basal ganglia circuitry.
During learning in juveniles, the relationship between
vocal exploration and eventual exploitation is more
obvious, as birds move from highly variable to
signicantly more stereotyped songs. In adults, recent
work indicates that the song variability continues to be
signicant for song plasticity. Trial-by-trial variability in
adult song can enable reinforcement learning, and the
degree of variability can constrain the amount of
plasticity (Tumer and Brainard, 2007; Andalman and
Fee, 2009; Charlesworth et al., 2011; Warren et al.,
2011; Sober and Brainard, 2012).

LOCUS FOR THE ACTIVE GENERATION OF

VARIABILITY
To date, many studies have focused on the output of the
AFP, LMAN, in investigating the role of this circuit in
providing both bias and variability to the motor pathway.
While this approach has been crucial in establishing a

role for the AFP in modifying motor output and enabling

plasticity, LMAN is the output of a larger circuit. Thus,
studies of LMAN alone leave open the question of
where and how variability is generated. In particular, is
variability a property of the cortical output nucleus or
does it arise elsewhere in the cortical-basal ganglia
circuitry?
Studies in juvenile birds show that lesions of the basal
ganglia nucleus Area X and lesions of the cortical outow
nucleus LMAN have distinct eects. In both cases,
juveniles fail to properly learn the tutor song (Schar
and Nottebohm, 1991). However, while lesions of LMAN
result in premature song stereotypy, Area X lesions result
in songs where both the sequence and the structure of
syllables remain variable (Schar and Nottebohm, 1991;
Goldberg and Fee, 2011). These data indicate that there
may be dierential contributions of these two nuclei to
sensorimotor learning in young birds.
In adults, lesions of Area X and LMAN both reduce
song plasticity, but, as is true in juveniles, they appear
to have dierent roles in modulating trial-by-trial
variability. In general, adult song plasticity has been
studied by altering auditory feedback. For example,
plasticity can be generated by removing, through
deafening, naturally occurring auditory feedback
(Nordeen and Nordeen, 1997; Brainard and Doupe,
2000; Kojima et al., 2013), and can be driven through

Please cite this article in press as: Woolley SC, Kao MH. Variability in action: Contributions of a songbird cortical-basal ganglia circuit to vocal motor
learning and control. Neuroscience (2014), http://dx.doi.org/10.1016/j.neuroscience.2014.10.010

S. C. Woolley, M. H. Kao / Neuroscience xxx (2014) xxxxxx

the imposition of distorted auditory feedback (Tumer and

Brainard, 2007; Sober and Brainard, 2009; Ali et al.,
2013). In both of these cases, feedback-driven changes
in syllable features, such as pitch, are inhibited by lesions
of either LMAN or Area X. However, although both LMAN
and Area X lesions inuence song plasticity, Area X
lesions only transiently reduce the degree of pitch variability in adult birds (Ali et al., 2013; Kojima et al., 2013)
whereas LMAN lesions cause long-lasting reductions in
the variability of pitch (Kao and Brainard, 2006;
Hampton et al., 2009). These data in both juveniles and
adults hint at potentially dierent roles for these nuclei in
generating variability that subserves plasticity (Fee and
Goldberg, 2011; Kojima et al., 2013).
However, because the AFP is a circuit, one
complication in interpreting the dierential eects of
LMAN and Area X lesions is that lesions of Area X likely
alter activity throughout the circuit. Indeed, recent data
indicate that eliminating activity in Area X alters the
activity of LMAN neurons (Kojima et al., 2013). Specically, after Area X lesions, LMAN is not silent but instead
continues to show both spontaneous activity and
increased ring with singing. However, this singingrelated ring shows a marked loss of patterned bursting,
indicating that burst ring is not an intrinsic property of
LMAN neurons but rather is a circuit property dependent
on activity in Area X. Moreover, in the absence of normal,
patterned bursting in LMAN, song does not deteriorate
following altered feedback (deafening), despite the
presence of tonic increases in singing-related activity.
Thus, Area X lesions eliminate both the typical patterned

bursting in LMAN and deafening-induced plasticity. This

raises the possibility that random variability may not be
sucient to induce plasticity; rather, targeted variability
or bursting may be necessary to enable plastic changes.
Moreover, it remains to be determined how the residual
LMAN activity following basal ganglia lesions contributes
to activity in RA and to song variability, in particular
whether tonic, unpatterned activity could account for the
continued song variability apparent after Area X lesions.
To directly investigate where and how socially
modulated variability is generated within the AFP, we
systematically recorded single neurons at multiple
stages in the circuit (Fig. 3A) in birds either while they
performed a courtship song for females or sang in
isolation (Woolley et al., 2014; Fig. 3). We found that, in
HVC, cells that project to Area X (HVCX) exhibit ring that
is highly precise in both the timing and number of spikes
and is not modulated by social context (Fig. 3B vs C).
These data indicate that the social modulation of variability seen in LMAN (Fig. 2E, F) is not present in the inputs to
Area X from the motor pathway. Individual SNs in Area X,
the rst step in the AFP, also exhibit precise song-locked
bursts that are equally reliable during directed and undirected singing (Fig. 3D vs. E). However, unlike HVCX neurons, SNs also show a small but consistent increase in the
mean ring rate during undirected singing. In contrast,
GPi neurons, the output cells of Area X, show dramatic
social context-dependent dierences in both ring rate
and spike timing across trials (Fig. 3F vs. G). This is most
apparent in pauses in ring, which are thought to reect
the inhibitory inputs of SNs onto GPi neurons and allow

Fig. 3. (A) Diagram of the song system. Subsequent panels show recordings from neurons in HVC (blue) and Area X (green). (B, C) In HVC,
neurons that project to Area X are highly precise and do not dier in the precision of spike timing or ring rate between social contexts. (D, E) The
ring precision of SNs in Area X is also not aected by social context, although the mean ring rate is slightly but signicantly higher during
undirected singing. (F, G) In contrast, GPi neurons in Area X display dramatic dierences in both the rate and variability of ring, with higher, more
variable activity during undirected singing. This social modulation does not rely on input from LMAN. Modied from Woolley et al. (2014).
Please cite this article in press as: Woolley SC, Kao MH. Variability in action: Contributions of a songbird cortical-basal ganglia circuit to vocal motor
learning and control. Neuroscience (2014), http://dx.doi.org/10.1016/j.neuroscience.2014.10.010

S. C. Woolley, M. H. Kao / Neuroscience xxx (2014) xxxxxx

Fig. 4. Illustration of one potential model for how a dierence in rate correlation among spiny neurons (SNs) could produce the substantial social
modulation of variability in GPi neurons. Convergence of SNs onto a single GPi neuron is depicted for ve SNs under a high dopamine (A) and low
dopamine (B) condition. (A) Firing rate for ve SNs across ve trials, with rate corresponding to color based on the heat map on the left. When
dopamine levels are high, as is seen during directed singing, the model assumes that ring rate is uncorrelated across SNs and the number of
spikes for each cell will be independent on a given trial (dierent colors across cells). For a GPi neuron receiving convergent inhibitory input from this
population of SNs, this will result in an average amount of inhibitory input that is similar across trials, indicated by the similar color across all trials for
the convergent input. (B) In contrast, when dopamine levels are lower, as is seen during undirected singing, the model assumes that all SNs active
at the same time in song will re the same number of spikes on a given trial (increased rate correlation), indicated by the same color across all cells.
Thus, the downstream GPi neuron will receive dierent levels of inhibition across trials, indicated by the dierent colors across trials for the
convergent input. (C) Raster plot of a simulated GPi neuron. When SN inputs are uncorrelated, the downstream GPi neuron displays consistent
pauses in ring across trials. (D) Raster plot of a simulated GPi neuron when the SN inputs are correlated. In this case, the downstream GPi neuron
displays pauses in activity that are less reliable or consistent across trials. Across 50 simulations, each involving 50 trials and 1000 SNs synapsing
onto a single GPi neuron, the average CC of pause onsets was signicantly higher and less variable when SN rates were uncorrelated than when
they were correlated (0.81 versus 0.67, respectively; p < 0.0001). These eects on GPi neuron pausing parallel the social modulation of variability
seen in actual GPi neurons. Modied from Woolley et al. (2014).

downstream thalamic neurons to re, either via post-inhibitory rebound or disinhibition (Person and Perkel, 2005,
2007; Kojima and Doupe, 2009; Goldberg and Fee,
2012). These pauses are less reliable and show greater
variation in timing from trial-to-trial during undirected than
directed singing. Thus, by recording at multiple stages in
the circuit, we found that between the input and output
of the basal ganglia, activity transitions from being highly
stereotyped to showing substantial trial-by-trial variability.
Area X GPi neurons receive inputs from SNs intrinsic
to Area X, HVCX neurons and collaterals from LMAN
neurons projecting to RA (Farries et al., 2005; Leblois
et al., 2009). While HVCX neurons and SNs show no
social modulation in spike-timing variability, LMAN ring
is strongly modulated by social context. Thus, contextdependent changes in trial-by-trial variability in GPi

neuron ring could either arise rst in the basal ganglia

or could originate later in the circuit and be fed back into
Area X from LMAN (Fig. 3A). In order to distinguish
between these two possibilities, we lesioned LMAN, thus
making the circuit completely feed forward, and recorded
the activity of pallidal neurons. Following lesions of LMAN,
the activity of GPi neurons in Area X remains variable and
sensitive to social context (Woolley et al., 2014). These
data imply that variability in GPi neurons does not derive
from LMAN or HVC inputs but rather is generated within
the basal ganglia proper, and raise the possibility that variability may arise specically in the connection between
SNs and GPi neurons.
To understand how the activity of SNs, which are
highly precise in all contexts, could give rise to the
variable and socially modulated activity of GPi neurons,

Please cite this article in press as: Woolley SC, Kao MH. Variability in action: Contributions of a songbird cortical-basal ganglia circuit to vocal motor
learning and control. Neuroscience (2014), http://dx.doi.org/10.1016/j.neuroscience.2014.10.010

S. C. Woolley, M. H. Kao / Neuroscience xxx (2014) xxxxxx

we propose one simple model incorporating the known

ring properties of these neurons, the presumed
convergence of many SNs onto pallidal neurons, and
potential dopamine-dependent dierences in rate
correlation based on data in mammals (Woolley et al.,
2014; Fig. 4). In mammals, dopamine release is known
to aect the degree of rate correlation among SNs, with
lower correlation among neurons during higher dopamine
states (Bergman et al., 1998; Goldberg et al., 2004; Costa
et al., 2006). In zebra nches, dopamine inuences context-dependent dierences in song, and is increased in
Area X during courtship singing (Sasaki et al., 2006;
Leblois et al., 2010). Based on this, we hypothesize that
context-dependent dierences in dopamine release could
modulate the degree of rate correlation between SNs in
Area X.
Specically, in our model, increases in dopamine
during directed singing, would result in uncorrelated
ring rates among SNs that re at the same point in
song. Such lower correlations in the ring rate of SNs
would result in levels of inhibition that are similar across
trials, and GPi neurons would pause more reliably in
response to the average ring of their SN inputs
(Fig. 4A). In contrast, during undirected singing,
dopamine levels are lower, and we hypothesize that SNs
would be more correlated in their activity during each
trial. In this condition, for each trial, all cells that are
active at a particular time in song would re the same
number of spikes on that trial (Fig. 4B). Thus, variation in
the number of spikes across trials would be the same for
all cells that re at that time in song. Given the variability
in ring rate across trials seen in single neurons,
increases in rate correlations among SNs would result in
greater variation in the degree of inhibition on GPi
neurons across trials and therefore in the GPi pauses.
Using a simulation, we found that simply by altering
the rate correlation of SNs we could produce dramatic
dierences in the reliability and variability of pallidal
neuron activity that parallel the dierences in activity
that we see in our recordings (Fig. 4C, D; Woolley et al.,
2014). Thus, this model provides one possible framework
for understanding how a highly variable signal in pallidal
neurons could be generated from SN inputs that display
highly precise spike timing. This hypothesis can be tested
directly in future studies that measure correlations across
SNs in dierent social contexts and manipulate levels of
dopamine in Area X. Moreover, it will be important to
examine the contributions of other striatal and pallidal cell
types within Area X to the social modulation of variability.
Finally, as these data have highlighted, the AFP is a circuit. While we have focused on Area X and LMAN, lesion
and electrophysiological studies of the thalamic nucleus
DLM hint that DLM is much more than just a simple relay
station and underscore the importance of understanding
how signals are altered in the thalamus and how they contribute to song performance, variability and plasticity
(Chen et al., 2014; Goldberg and Fee, 2011, 2012;
Goldberg et al., 2012). Addressing these and other future
directions will provide needed insight into understanding
the generation of variability within cortical-basal ganglia
circuitry.

CONCLUSION
Understanding the learning and execution of behavior is a
fundamental question in neuroscience. Because
songbirds possess a cortical-basal ganglia circuit
specic for a single behavior, song, they oer a unique
model in which to investigate the circuits important for
generating and modulating a learned behavior.
Recently, studies on songbirds have been crucial to
informing our current understanding of the function and
generation of variability in learning and plasticity. Work
to date, in both mammals and songbirds, indicates that
the basal ganglia may be a locus where the three
components instrumental to reinforcement learning are
integrated: motor exploration or variability, error and
evaluation signals, and biasing signals to adjust future
actions. Future studies investigating how these three
components are generated, interact and are propagated
will be critical to further our understanding of motor
learning, motor control and motor dysfunction, and
songbirds promise to remain an essential model for
studying that integration.

REFERENCES
Ali F, Otchy TM, Pehlevan C, Fantana AL, Burak Y, Olveczky BP
(2013) The basal ganglia is necessary for learning spectral, but
not temporal, features of birdsong. Neuron 80:494506.
Andalman AS, Fee MS (2009) A basal ganglia-forebrain circuit in the
songbird biases motor output to avoid vocal errors. Proc Natl
Acad Sci USA 106:1251812523.
Barnes TD, Kubota Y, Hu D, Jin DZ, Graybiel AM (2005) Activity of
striatal neurons reects dynamic encoding and recoding of
procedural memories. Nature 437:11581161.
Bar Gad I, Bergman H (2001) Stepping out of the box: information
processing in the neural networks of the basal ganglia. Curr Opin
Neurobiol 11:689695.
Bergman H, Feingold A, Nini A, Raz A, Slovin H, Abeles M, Vaadia E
(1998) Physiological aspects of information processing in the
basal ganglia of normal and parkinsonian primates. TINS
21:3238.
Bottjer SW, Miesner EA, Arnold AP (1984) Forebrain lesions disrupt
development but not maintenance of song in passerine birds.
Science 224:901903.
Bottjer SW (1993) The distribution of tyrosine hydroxylase
immunoreactivity in the brains of male and female zebra
nches. J Neurobiol 24:5169.
Brainard MS, Doupe AJ (2002) What songbirds teach us about
learning. Nature 417:351358.
Brainard MS, Doupe AJ (2000) Interruption of a basal gangliaforebrain circuit prevents plasticity of learned vocalizations.
Nature 404:762766.
Brasted PJ, Wise SP (2004) Comparison of learning-related neuronal
activity in the dorsal premotor cortex and striatum. Eur J Neurosci
19:721740.
Carrillo G, Doupe AJ (2004) Is the songbird Area X striatal, pallidal, or
both? An anatomical study. J Comp Neurol 473:415437.
Charlesworth JD, Tumer EC, Warren TL, Brainard MS (2011)
Learning the microstructure of successful behavior. Nat
Neurosci 14:373380.
Chen JR, Stepanek L, Doupe AJ (2014) Dierential contributions of
basal ganglia and thalamus to song initiation, tempo, and
structure. J Neurophysiol 111:248257.
Churchland MM, Yu BM, Ryu SI, Santhanam G, Shenoy KV (2006a)
Neural variability in premotor cortex provides a signature of motor
preparation. J Neurosci 26:36973712.

Please cite this article in press as: Woolley SC, Kao MH. Variability in action: Contributions of a songbird cortical-basal ganglia circuit to vocal motor
learning and control. Neuroscience (2014), http://dx.doi.org/10.1016/j.neuroscience.2014.10.010

S. C. Woolley, M. H. Kao / Neuroscience xxx (2014) xxxxxx

Churchland MM, Afshar A, Shenoy KV (2006b) A central source of

movement variability. Neuron 52:10851096.
Costa RM, Lin SC, Sotnikova TD, Cyr M, Gainetdinov RR, Caron MG,
Nicolelis MA (2006) Rapid alterations in corticostriatal ensemble
coordination
during
acute
dopamine-dependent
motor
dysfunction. Neuron 52:359369.
DeLong MR (1971) Activity of pallidal neurons during movement. J
Neurophysiol 34:414427.
DeLong MR, Wichmann T (2007) Circuits and circuit disorders of the
basal ganglia. Arch Neurol 64:2024.
Ding L, Perkel DJ (2002) Dopamine modulates excitability of spiny
neurons in the avian basal ganglia. J Neurosci 22:52105218.
Doupe AJ, Perkel DJ, Reiner A, Stern EA (2005) Birdbrains could
teach basal ganglia research a new song. Trends Neurosci
28:353363.
Doya K (2002) Metalearning and neuromodulation. Neural Netw
15:495506.
Doya K, Sejnowski TJ (1998) A computational model of birdsong
learning by auditory experience and auditory feedback. In: Poon
PWF, Brugge JF, editors. Central auditory processing and neural
modeling. New York, NY: Plenum. p. 7788.
Doya K, Sejnowski TJ (2000) A computational model of avian song
learning. In: Gazzaniga MF, editor. The new cognitive
neurosciences. Cambridge, MA: MIT. p. 469482.
Dugas-Ford J, Rowell JJ, Ragsdale CW (2012) Cell-type homologies
and the origins of the neocortex. Proc Natl Acad Sci USA
109:1697416979.
Farries MA, Perkel DJ (2002) A telencephalic nucleus essential for
song learning contains neurons with physiological characteristics
of both striatum and globus pallidus. J Neurosci 22:37733787.
Farries MA, Ding L, Perkel DJ (2005) Evidence for direct and
indirect pathways through the song system basal ganglia. J
Comp Neurol 484:93104.
Fee MS, Goldberg JH (2011) A hypothesis for basal gangliadependent reinforcement learning in the songbird. Neuroscience
198:152170.
Gale SD, Person AL, Perkel DJ (2008) A novel pathway forms a loop
linking a vocal learning circuit with its dopaminergic input. J Comp
Neurol 508:824839.
Gdowski MJ, Miller LE, Parrish T, Nenonene EK, Houk JC (2001)
Context dependency in the globus pallidus internal segment
during targeted arm movements. J Neurophysiol 85:9981004.
Goldberg JH, Adler A, Bergman H, Fee MS (2010) Singing-related
neural activity distinguishes two putative pallidal cell types in the
songbird basal ganglia: comparison to the primate internal and
external pallidal segments. J Neurosci 30:70887098.
Goldberg JH, Farries MA, Fee MS (2012) Integration of cortical and
pallidal inputs in the basal ganglia-recipient thalamus of singing
birds. J Neurophysiol 108:14031429.
Goldberg JH, Fee MS (2010) Singing-related neural activity
distinguishes four classes of putative striatal neurons in the
songbird basal ganglia. J Neurophysiol 103:20022014.
Goldberg JH, Fee MS (2011) Vocal babbling in songbirds requires the
basal ganglia-recipient motor thalamus but not the basal ganglia.
J Neurophysiol 105:27292739.
Goldberg JH, Fee MS (2012) A cortical motor nucleus drives the
basal ganglia-recipient thalamus in singing birds. Nat Neurosci
15:620627.
Goldberg JA, Rokni U, Boraud T, Vaadia E, Bergman H (2004) Spike
synchronization in the cortex/basal-ganglia networks of
Parkinsonian primates reects global dynamics of the local eld
potentials. J Neurosci 24:60036010.
Graybiel AM (2005) The basal ganglia: learning new tricks and loving
it. Curr Opin Neurobiol 15:638644.
Graybiel AM (2008) Habits rituals and the evaluative brain. Annu Rev
Neurosci 31:359387.
Graybiel AM, Aosaki T, Flaherty AW, Kimura M (1994) The basal
ganglia and adaptive motor control. Science 265:18261831.
Grillner S, Hellgren J, Menard A, Saitoh K, Wikstrom MA (2005)
Mechanisms for selection of basic motor programs roles for the
striatum and pallidum. TINS 28:364370.

Hampton CM, Sakata JT, Brainard MS (2009) An avian basal gangliaforebrain circuit contributes dierentially to syllable versus
sequence variability of adult Bengalese nch song. J
Neurophysiol 101:32353245.
Handel A, Glimcher PW (2000) Contextual modulation of substantia
nigra pars reticulata neurons. J Neurophysiol 83:30423048.
Hessler NA, Doupe AJ (1999) Social context modulates singingrelated neural activity in the songbird forebrain. Nat Neurosci
2:209211.
Hikosaka O, Sakamoto M, Usui S (1989) Functional properties of
monkey caudate neurons I Activities related to saccadic eye
movements. J Neurophysiol 61:780798.
Immelman K (1969) Song development in the zebra nch and other
estrildid
nches.
In:
Hinde
RA,
editor.
Bird
vocalizations. London: Cambridge Univ Press. p. 6174.
Ishii S, Yoshida W, Yoshimoto J (2002) Control of exploitationexploration meta-parameter in reinforcement learning. Neural
Netw 15:665687.
Jarvis ED, Gunturkun O, Bruce L, Csillag A, Karten H, Kuenzel W,
Medina L, Paxinos G, Perkel DJ, Shimizu T, Striedter G, Wild JM,
Ball GF, Dugas-Ford J, Durand SE, Hough GE, Husband S,
Kubikova L, Lee DW, Mello CV, Powers A, Siang C, Smulders TV,
Wada K, White SA, Yamamoto K, Yu J, Reiner A, Butler AB,
Avian Brain Nomenclature Consortium (2005) Avian brains and a
new understanding of vertebrate brain evolution. Nat Rev
Neurosci 6:151159.
Kao MH, Brainard MS (2006) Lesions of an avian basal ganglia circuit
prevent context-dependent changes to song variability. J
Neurophysiol 96:14411455.
Kao MH, Doupe AJ, Brainard MS (2005) Contributions of an avian
basal ganglia-forebrain circuit to real-time modulation of song.
Nature 433:638643.
Kao MH, Wright BD, Doupe AJ (2008) Neurons in a forebrain nucleus
required for vocal plasticity rapidly switch between precise ring
and variable bursting depending on social context. J Neurosci
28:1323213247.
Karten HJ (1969) The organization of the avian telencephalon and
some speculations on the phylogeny of the amniote
telencephalon. In: Pertras J, editor. Comparative and
evolutionary aspects of the vertebrate central nervous
system. New York: Ann N Y Acad Sci. p. 146179.
Karten HJ (1991) Homology and evolutionary origins of the
neocortex. Brain Behav Evol 38:264272.
Kawagoe R, Takikawa Y, Hikosaka O (1998) Expectation of reward
modulates cognitive signals in the basal ganglia. Nat Neurosci
1:411416.
Kojima S, Doupe AJ (2009) Activity propagation in an avian basal
ganglia-thalamocortical circuit essential for vocal learning. J.
Neurosci. 29:47824793.
Kojima S, Doupe AJ (2011) Social performance reveals unexpected
vocal competency in young songbirds. Proc Natl Acad Sci USA
108:16871692.
Kojima S, Kao MH, Doupe AJ (2013) Task-related cortical bursting
depends critically on basal ganglia input and is linked to vocal
plasticity. Proc Natl Acad Sci USA 110:47564761.
Konishi M (1965) The role of auditory feedback in the control of
vocalization in the whitecrowned sparrow. Z Tierpsychol
22:770783.
Lauwereyns J, Watanabe K, Coe B, Hikosaka O (2002) A neural
correlate of response bias in monkey caudate nucleus. Nature
418:413417.
Leblois A, Bodor AL, Person AL, Perkel DJ (2009) Millisecond
timescale disinhibition mediates fast information transmission
through an avian basal ganglia loop. J Neurosci 29:1542015433.
Leblois A, Wendel BJ, Perkel DJ (2010) Striatal dopamine modulates
basal ganglia output and regulates social context-dependent
behavioral variability through D1 receptors. J Neurosci
30:57305743.
Lewis JW, Ryan SM, Arnold AP, Butcher LL (1981) Evidence for a
catecholaminergic projection to area X in the zebra nch. J Comp
Neurol 196:347354.

Please cite this article in press as: Woolley SC, Kao MH. Variability in action: Contributions of a songbird cortical-basal ganglia circuit to vocal motor
learning and control. Neuroscience (2014), http://dx.doi.org/10.1016/j.neuroscience.2014.10.010

S. C. Woolley, M. H. Kao / Neuroscience xxx (2014) xxxxxx

Luo M, Perkel DJ (1999) A GABAergic strongly inhibitory projection to
a thalamic nucleus in the zebra nch song system. J Neurosci
19:67006711.
Marler P (1970) A comparative approach to vocal learning: song
development in white crowned sparrows. J Comp Physiol Psychol
7:1125.
Mooney R (2009) Neurobiology of song learning. Curr Opin
Neurobiol. 19:654660.
Nordeen KW, Nordeen EJ (1997) Anatomical and synaptic substrates
for avian song learning. J Neurobiol. 33:532548.
Nottebohm F, Stokes TM, Leonard CM (1976) Central control of song
in the canary Serinus canarius. J Comp Neurol 165:457486.
Olveczky BP, Andalman AS, Fee MS (2005) Vocal experimentation in
the juvenile songbird requires a basal ganglia circuit. PLoS Biol
3:e153.
Osborne LC, Lisberger SG, Bialek W (2005) A sensory source for
motor variation. Nature 437:412416.
Pasupathy A, Miller EK (2005) Dierent time courses of learningrelated activity in the prefrontal cortex and striatum. Nature
433:873876.
Person AL, Perkel DJ (2005) Unitary IPSPs drive precise thalamic
spiking in a circuit require for learning. Neuron 46:129140.
Person AL, Perkel DJ (2007) Pallidal neuron activity increases during
sensory relay through thalamus in a songbird circuit essential for
learning. J Neurosci 27:86878698.
Person AL, Gale SD, Farries MA, Perkel DJ (2008) Organization of
the songbird basal ganglia including area X. J Comp Neurol
508:840866.
Reiner A, Laverghetta AV, Meade CA, Cuthbertson SL, Bottjer SW
(2004) An immunohistochemical and pathway tracing study of the
striatopallidal organization of area X in the male zebra nch. J
Comp Neurol 469:239261.
Reiner A, Yamamoto K, Karten HJ (2005) Organization and evolution
of the avian forebrain. Anat Rec A Discov Mol Cell Evol Biol
287:10801102.
Sakata JT, Brainard MS (2009) Social context rapidly modulates the
inuence of auditory feedback on avian vocal motor control. J
Neurophysiol 102:24852497.
Sasaki A, Sotnikova TD, Gainetdinov RR, Jarvis ED (2006) Social
context-dependent singing regulated dopamine. J Neurosci
30:90109014.
Schar C, Nottebohm F (1991) A comparative study of the behavioral
decits following lesions of various parts of the zebra nch song
system: implications for vocal learning. J Neurosci 11:28962913.

Schultz W (2013) Updating dopamine reward signals. Curr Opin

Neurobiol 23:229238.
Schultz W, Tremblay L, Hollerman JR (2003) Changes in behaviorrelated neuronal activity in the striatum during learning. Trends
Neurosci 26:321328.
Sheth SA, Abuelem T, Gale JT, Eskander EM (2011) Basal ganglia
neurons dynamically facilitate exploration during associative
learning. J Neurosci 31:48784885.
Sober SJ, Brainard MS (2009) Adult birdsong is actively maintained
by error correction. Nat Neurosci. 12:927931.
Sober SJ, Brainard MS (2012) Vocal learning is constrained by the
statistics of sensorimotor experience. Proc Natl Acad Sci USA
109:2109921103.
Soha JA, Shimizu T, Doupe AJ (1996) Development of the
catecholaminergic innervation of the song system of the male
zebra nch. J Neurobiol 29:473489.
Stepanek L, Doupe AJ (2010) Activity in a cortical-basal ganglia
circuit for song is required for social context-dependent vocal
variability. J Neurophysiol 104:24742486.
Sutton RS, Barto AG (1998) Reinforcement learning: an
introduction. Cambridge, MA: MIT Press.
Tumer EC, Brainard MS (2007) Performance variability enables
adaptive plasticity of crystallized adult birdsong. Nature
450:12401244.
Warren TL, Tumer EC, Charlesworth JD, Brainard MS (2011)
Mechanisms and time course of vocal learning and
consolidation in the adult songbird. J Neurophysiol
106:18061821.
Wichmann T, DeLong MR, Guridi J, Obeso JA (2011) Milestones in
research on the pathophysiology of Parkinsons disease. Mov
Disord 26:10321041.
Williams ZM, Eskandar EN (2006) Selective enhancement of
associative learning by microstimulation of the anterior caudate.
Nat Neurosci 9:562568.
Woolley SC, Doupe AJ (2008) Social context-induced song
variation aects female behavior and gene expression. PLoS
Biol 6:e62.
Woolley SC, Rajan R, Joshua M, Doupe AJ (2014) Emergence of
context-dependent variability across a basal ganglia circuit.
Neuron 82:208223.
Wu HG, Miyamoto YR, Gonzalez Castro LN, Olveczky BP, Smith MA
(2014) Temporal structure of motor variability is dynamically
regulated and predicts motor learning ability. Nat Neurosci.
17:312321.

(Accepted 8 October 2014)

(Available online xxxx)

Please cite this article in press as: Woolley SC, Kao MH. Variability in action: Contributions of a songbird cortical-basal ganglia circuit to vocal motor
learning and control. Neuroscience (2014), http://dx.doi.org/10.1016/j.neuroscience.2014.10.010