Eddie Prez Claudio

Reversal Learning
Well known for his experiments in classical conditioning, Ivan Pavlov was among
the first to directly study learning in a quantitative and qualitative manner. It was thanks
to his research that we know animals possess different learning methodologies by
which they may permanently or temporarily modify their behavior. This has a direct
impact on the survival rate and adaptability (fitness) of an organism, be it as an
individual or as a group.
Through his studies Pavlov found there are two phases in learning:
Acquisition: The process by which an external stimulus initiates the formation of a new
memory that may or may not have a behavioral response. In classical conditioning it is
the process by which a neutral stimulus (NS) is paired with an unconditioned stimulus
(US) to elicit an unconditioned response (UR). Once acquisition has occurred the
animal will elicit the UR when the NS is presented without the US. By this point the NS
becomes a conditioned stimulus (CS) and the UR a conditioned response (CR).
Example: The sound mom has no initial meaning, it is through the
parents act of repeating the word as they point to themselves that the
baby then acquires the meaning of the word mom.
Extinction: The process by which a memory is suppressed and the subject becomes
unable to recall it. This effect can be short-term or long-term. In classical conditioning it
is the process by which a CS becomes an NS. Studies that focus on this process call it
extinction learning.
Example: A dog that has been trained to roll on command will
eventually stop doing so if the command is not followed by the
expected reward (a pat on the head or a treat).
In humans and other animals, researchers have studied different types of
learning, all of which undergo the same basic principles of acquisition and extinction.
These different types of learning are not exclusive, (i.e. a single test may have multiple
interactions), and they are grouped in many different ways. They may be grouped in
relation to the capabilities required to learn: Sensory (additive, visual, kinetic),
Holistic, or Action-oriented learning; each with its own subgroup. Grouping learning
types in this manner is common in pedagogy where it has been found humans have
preferences in the method by which they learn. Some prefer to listen to a lecture rather
than reading a book (Sensory), others will learn by trying out the theory (Actionoriented), and another group prefers a mix of both (Holistic). In experimental designs, to
avoid confusion, it is more common to group learning by the nature of the test or assay
the subject is going through. Aversive, Appetitive, Discrimination, Operant, Place
Preference, Olfactory, Habituation, and Sensitization learning paradigms are among the
most common of the categories, although, other fields may refer to these using
synonyms.

All of these learning strategies partake on improving the process of adaptation,

especially discrimination learning. During this type of learning, the subject will be
presented with two CSs. One CS will be paired with the US (CS+) and the other will not
(CS-). The subject will learn two things: (1) to elicit the CS with the CS+ and (2) to inhibit
the CR in the presence of the CS-. A clear example is the honeybee. In the wild,
honeybee nectar foragers need to choose from a wide array of flowers which one it will
visit, they usually prefer the ones with easy access and high sucrose (sugar)
concentration. However, in cities where flowers are uncommon, they had to adapt. Now
it is common to see foragers flying around soda pops in trashcans, seeking sucrose to
maintain the hive. Since the reward (sucrose) they get from the soda is higher than that
of a flower and the risk is usually lower, honeybees are making the shift. This is an
example of appetitive and discrimination learning.
To be effective learning processes must induce memory consolidation (formation).
Memory consolidation has various stages which can be used to classify memory. These
stages are Immediate, Short-Term, and Long-Term. Most of the memories studied by
researchers are either Short-Term or Long-Term. Memories formed during a learning
session will also vary in type depending on the learning task. They may be Declarative
(language and history) or Non-declarative (motor skills and associations). Thus learning
paradigms influence where, how, and for how long memories are processed. During
decision-making memories are all weighted to produce a behavioral response. Therein
lies our first clue about Reversal learning.
Decision-making and reversal learning are biologically linked to each other.
Mammals have neural circuits that overlap in the ventral prefrontal cortex (vPFC) and
circuits in the orbitofrontal cortex (OFC) take part in both processes. However, not
everything is the same. Studies have shown that mental disorders linked to the PFC can
be studied more selectively through reversal learning rather than using decision-making
as the experimental variable. Currently thousands of researchers conduct reversal
learning tests to study ADHD, Addiction, OCD, and Schizophrenia. Moreover, reversal
learning also has an impact on ecological studies.
Recent findings show that honeybee subspecies have differences in their reversal
learning capabilities. These differences can be correlated with their endemic habitats.
Making even clearer the importance of learning, especially reversal learning, for
environmental adaptation. In the case of foragers this adaptation would be translated
into flower visit patterns, which are of utmost importance when studying honeybee
ecology.
First studied by Ivan Pavlov in the 1920s, reversal learning is commonly defined
as the ability to learn an unrewarded stimulus from an initial discrimination paradigm.
For his experiments, Pavlov trained dogs to salivate at the sound of a strong tone by
giving them food every time it was rung. Conversely he trained them to do nothing when
a weak tone was rung. After the conditioning had taken place, he reversed the
experimental paradigm and found the association between the strong and weak tones
was reversed.
Since this sounds confusing, lets make an example. Remember learning types can be
mixed and usually have multiple interactions. If we take some honeybee foragers and

mark them so we can tell them apart. Now place an artificial flower and an empty soda
can side by side in a controlled environment (your yard) and train them so they will visit
both the flower and the can. To begin the experiment place sucrose (US) on the artificial
flower, this will be the CS+ and clean the soda can the CS-. Follow the foragers for a
while a record which CS they visited. This is your average appetitive discrimination
learning test. After some time, clean the artificial flower so it has no sucrose (CS-) and
put some in the soda can (CS+). We have now reversed the paradigm and the foragers
will start appetitive discrimination reversal learning.
During reversal learning, Acquisition and Extinction become ambiguous and
occur simultaneously. In order for the new memory to be consolidated, the old one is
suppressed, so that it does not interfere with learning. This is achieved by maintaining
memory destabilization, which causes the memory to be inaccessible. During reversal,
a temporary suppression of reconsolidation (stabilization) causes an Extinction like
effect. For the Acquisition of the reversal memory, an additional brain area is used to
regulate memory formation. All of this makes reversal learning a highly taxing and
evolutionarily important mental process. Thus initial acquisition will not necessarily
predict future acquisition which is often impaired.
Reversal learning should not be confused with Reverse learning. Reverse
learning was first proposed by Francis Crick and Graeme Mitchison on a Nature article
in 1983. They proposed a mechanism that occurs during rapid eye movement (REM)
sleep that weakens the connections between synapses rather than strengthening them.
Reverse learning is then a process contrary to long-term potentiation (LTP) or long-term
memory consolidation. On the other hand, reversal learning is focused on changes to
the reward consistencies in a differential learning paradigm.
Even though reversal learning paradigms have been popular since the 1950s, research
as to how its molecular mechanism works is still scarce. This has to do with the
fact that most studies that are related to reversal learning, use it as a tool rather
than an end.
For more information read:
Almeida-Corra S, A. O. (2014). Memory labilization in reconsolidation and
extinction--evidence for a common plasticity system? J Physiol Paris, 292-306.;
l. Clark, R. C. (2004). The neuropsychology of ventral prefrontal cortex: Decisionmaking and reversal learning. Brain and Cognition, 41-53;
Izquierdo, A., & Jentsch, J. D. (2012). Reversal learning as a measure of
impulsive and compulsive behavior in addictions.Psychopharmacology,219 (2),
607620. doi:10.1007/s00213-011-2579-7;
Xue G, X. F. (2013). Common Neural Mechanisms Underlying Reversal Learning
by Reward and Punishment. PLoS One.