Complex Neural Networks – A Useful

Model of Human Learning?

Mark Hardman

Canterbury Christ Church University

mark.hardman@canterbury.ac.uk

Presented at:

British Educational Research Association Annual conference

University of Warwick

2nd September, 2010

Page | 1
Artificial neural networks are computational systems which are increasingly common and
sophisticated. Computational scientists who work with such systems however, often assume that
they are simplistic versions of the neural systems within our brains and Cilliers (1998) has gone
further in proposing that human learning takes place through the self-organisation of such systems
according to the stimuli they receive. This continues a long tradition of using metaphors to
machinery in order to describe the human brain and its processes. Unlike other metaphors
however, artificial neural networks were originally conceived of in relation to how biological neurons
function. As such, does this metaphor allow us to accurately understand brain function? In this
paper I wish to consider the level of support this model has from contemporary neuroscience and in
doing so evaluate the possible implications for educationalists.

A complex neural network may be considered as a series

of nodes connected by neurons such that they are highly Figure 1 – A simple neural network
interconnected. The use of the term complex here
denotes that each node is only responding to signals from
the neurons it is directly connected to, yet the system as a
whole is able to respond to its environment.

Such artificial neural networks are able to respond because

they adapt according to Hebb’s law. This states that the
more a neural path between two nodes is used, the more
efficient that path becomes. As such, constant input to a
neural system results in what Edelman (1987) referred to
as ‘neural Darwinism’: a neural system adapts according to
input stimulus making the used pathways more effective within it.

Artificial neural networks differ from the majority of modern day computers which utilise what is
denoted as Von Neumann architecture. Computers, such as the one I am using to present today,
step through rules and processes in sequence. Even though contemporary quad core computer
processors divide up tasks so rules can be stepped through by four processors at once, the system is
still stepping through a set of rules to respond to input. Neural networks do not have rules or
predetermined programs, they simply respond according to the internal structure of the system.

This makes neural networks much better than traditional computers at recognising and responding
to patterns and as such they have applications in number plate recognition (Draghici, 1997), sales
and marketing forecasts (Kuo & Xue, 1999) and image processing. Johnson & Hogg (1995) show
how a system can be trained to predict where a person will walk after observing people walking
through a scene a number of times1.

Cilliers (1998) discusses an example by which a neural network is trained to predict past tense verbs
given the present tense verb. In order to train such a system, it is presented with an input, and the
system is manually adjusted by changing the path strengths until the desired output is given. This is
repeated with a variety of inputs, in this case verbs. After this training phase the system is able to
deal with unfamiliar input and continues to adapt to the input it is presented with.

1
I shall be presenting videos of their system in operation.

Page | 2
His account goes on to discuss the requirement that the training input is sufficiently varied,
otherwise the neural system will be able to recognise only very specific patterns, for example a
specific tree, rather than all trees. For face recognition for example, the system must be trained
using pictures of the face in varied light and from various angles in order to effectively ‘learn’ to
recognise it.

Of course, we need to establish whether human brains actually work in this way before we can
ascertain the usefulness of complex neural network models to the field of education. Whilst
artificial neural networks were originally developed in line with an understanding of how neurons in
the brain work, there have, since that time, been advances in both neuroscience and artificial
networks. However, I think it would be fruitful to first hypothesise with respect to what neural
network models might tell us about human learning before we see to what level our assertions are
supported. This will provide an appreciation of the possibilities of neural network models within
education, which we can then rationalise according to the strength of evidence within the scientific
literature.

The first thing to note is the very different way learning is conceived of within educational theory
and within neural modelling (both computational and biological). Traditional notions within
education of learning as the acquisition of facts or skills are at odds with the interpretation of
learning as a process within a complex system. Both Kelso (1995) and Cilliers (1998) conceive of
learning as the reorganisation of the whole system. There are no structures or representations
within the network which correspond to specific knowledge or abilities. Cilliers (1998) denotes this
as “distributed representation”: the whole system is involved in the response to stimuli. If we accept
this then we also need to recognise that it is not just the information or task that a neural system is
responding to, but the stimuli of the entire environment. Furthermore as well as the external
environment, the internal structure at any given time is important because this determines the
starting point for any reorganisation. As educationalists then this model suggests that we need to
conceptualise learning as the interaction of the complex neural network within the mind and the
entire classroom environment.

What is of great interest to me is the suggestion from complex neural models that the brain learns to
respond to familiar contexts. Elsewhere2 I argue that we can consider a classroom itself as a
complex system and if this is the case then pupils are learning to respond to classroom systems. As
such they will learn behavioural norms within specific classrooms and the learning within each
classroom will be determined by the social dynamics of that classroom.

Beyond the reconceptualisation of learning, Cilliers’ account of the need for varied training examples
suggests that repetition is important, but also that presented contexts must be varied. This begs the
question of whether our curricula or the contexts we use in the classroom allow for learning in this
way. It would also be interesting to see what light this neurological model sheds on many
constructivist and behaviourist learning theories as it seems to suggest a need for repetition but with
variety of context and consideration of the history of individuals, although this is beyond the scope
of this paper.

2 th
“Learning to Teach in ‘Urban Complex Schools’” to be presented Saturday 4 September in Session 7.

Page | 3
I hope you agree that there are a range of very interesting hypotheses that can be generated by
considering learning in complex neural networks. We now need to consider current neuroscientific
understanding in order to ascertain whether any of these hypotheses may be supported.

Firstly let’s consider Hebb’s law which, although being a tenet of neuroscience since the 1950s, still
has not accumulated a large amount of scientific evidence to support it. This is due to the
experimental difficulties of seeing neural development within a living creature. Work by Antonov et
al (2003) with Aplysia (a type of sea slug) does however support the development of specific
pathways according to stimulus. Sylwester (1995) suggests that the mechanism for this is the
development of dendrites: projections from the neural cell that allow more neurotransmitter to
cross synapses, thus making the pathway more efficient. There is less certainty about what reduces
path strengths if neurons are not fired together but it is a modelling assumption of neural networks
that this must also happen.

Further support comes from work on Lampreys (primitive jawless fish), which have long and simple
neural networks which are conducive to investigation (Kelso, 1995). The networks can be observed
and replicated with artificial networks in order to study their adaptation.

Support for the need of repeated yet varied learning contexts comes primarily from the cognitive
sciences and fits with contemporary view of memory development. This suggests that initially we
have episodal memories of specific circumstances; then further experience leads to semantic
memory which can be applied in new situations; then eventually procedural memory develops in
which we are able to perform tasks subconsciously. Sylwester (1995) describes learning to type as
initially being about the specific typewriter he used, then his ability to type on any keyboard and
finally his ability to type without looking at a keyboard at all.

Despite the above tentative support however, there are a number of important limitations to the
complex neural model which are highlighted by reference to neuroscience. The most apparent
obstacle to either supporting or denying this model of brain development is the fact that the
majority of contemporary neuroscience does not deal with individual neurons, or even neural
systems, but instead focuses on the specific areas or modules within the brain. This is not because
neural networks do not exist but because we currently lack the technology to track neural activity to
that resolution.

What can be drawn from the focus on brain areas however is that if the same brain areas
correspond to certain behaviours in all studied humans, then the structure of the brain must be
determined by biological processes emanating from our genetics and the brain cannot be considered
as a system that is fully plastic with respect to stimuli. Cilliers (1998) recognises this and draws on
Edelman (1987) and Changeaux (1984) in describing how the brain must have a first repertoire of
structural organisation and a second repertoire of adaptation through experience. The interaction of
genetics and experience as well as determining the level of plasticity in the brain presents significant
experimental issues and conclusions are a long way off yet.

Another limitation of the complex neural systems model comes from the recognition within the
cognitive sciences that a great deal of the way we respond to the world is part of what makes us
human and cannot be explained by a model of the brain as fully plastic. Geake (2009) notes the
importance of non-Hebbian learning such as when a life changing event occurs and we quickly adapt

Page | 4
to it, or more subtly our ability to learn by hearing something only once (in most adults at least).
Moreover the ability of humans to copy the actions of another person just by seeing them and
without the need to experience that action ourselves suggests this ability owes more to biology than
society. This along with our ability to use language, imagine the future and everything else that
makes us human suggests that if we are to use neural networks as a model of how the brain works
we must somehow incorporate it into current understanding of brain development through nature
and nurture.

Despite these limitations, technological advances within neuroscience are making headway in
understanding how neural networks might function (Bressier & Menon, 2010; Bullmore & Sporns,
2009). In what are known as large-scale brain network models, specific areas of the brain are
treated as nodes and their interaction investigated. What constitutes a node can be defined in a
number of different ways according to a range of properties of the brain (e.g. nodes are areas within
a specific biochemical makeup or neuron density) and this yields a range of different models. By to
some extent ignoring the internal processes of the nodes, the interactions between the nodes can
be modelled in a similar way to the computational neural networks presented above. Whilst
cautioning that the isolation of specific neural networks in this way is still an inaccurate
representation of brain function Bressier & Menon (2010) argue that

“A new paradigm is emerging in cognitive neuroscience that moves beyond the simplistic
mapping of cognitive constructs onto individual brain areas and emphasizes instead the
conjoint function of brain areas working together as large-scale networks.”

There has already been promising work in identifying core functional brain networks (dealing with
spatial attention, language, explicit memory, face-object recognition and working memory) and in
relating these to disease and dementia.

So how are we to judge the hypotheses we developed in relation to neural networks and learning?
The simple answer is that neuroscience is not currently sophisticated enough to be able to judge
whether or not human learning takes place in this way. What is evident however is that the model
of complex neural networks developed in artificial networks does not account for the evolutionary
history of the human brain. As educationalists wait for, and hopefully contribute to, the
development of neurological understanding we must continue to draw on the cognitive sciences in
developing pedagogies and learning theories, whilst being open to the insights neuroscience may
provide. In short we may use neuroscientific understanding to develop hypothesises but we must
continue to test these using the full range of methods available to social scientists.

The most interesting hypotheses in my opinion are the reconceptualisation of learning as a complex
adaptation and the need for varied yet repeated examples in teaching. Much of my PhD thesis will
centre on looking at how pupils interact with their environment and peers in a classroom through
the lens of complex adaptive systems. I feel the need for repeated yet varied learning activities is
something that could more easily be tested through comparing students who are taught in this way
with a control group. This exemplifies the role neuroscientific models of learning may have in
providing social scientists with falsifiable hypothesises.

In conclusion then complex, neural networks draw on computer science to develop a model of the
way learning takes place in the brain. Whilst there is some support for this from the natural and

Page | 5
cognitive sciences it is in no way conclusive due to experimental difficulties at present. Nevertheless
this model offers a range of interesting hypotheses which might be tested by social scientists as
generating interesting theoretical discussion which will only become more interesting as the
neuroscientific underpinning continues to advance.

Antonov I., Antonova I., Kandel E.R. & Hawkins R.D. (2003) Activity-dependent presynaptic
facilitation and hebbian LTP are both required and interact during classical conditioning in
Aplysia. Neuron. 2003 Jan 9;37(1):135-47.

Bressler, S. L. & Menon, V. (2010) Large-scale brain networks in cognition: emerging methods
and principles. Trends in Cognitive Sciences, 14:277-290

Bullmore, E. & Sporns, O. (2009) Complex brain networks: graph theoretical analysis of structural
and functional systems. Nature, Vol. 10, March, pp186-195

Cilliers, P. (1998) Complexity and postmodernism: understanding complex systems, London:

Routledge.Feeman, J. (1999) How Brains Make Up Their Minds. London: Weidenfeld & Nicolson

Changeaux, J.-P., Heidemann, T. And Patte, P. (1984) Learning by Selection in Marler, P. &
Terrace (eds), The Biology of Learning. Berlin: Springer-Verlag. [Cited in Cilliers (1998)]

Draghici, S. (1997) A neural network based artificial vision system for licence plate recognition.
International Journal of Neural Systems, February, 8(1):113-126

Edelman, G.E. (1987) Neural Darwinism: The Theory of Neuronal Group Selection. New York:
Basic Books

Geake, J. G. (2009) The Brain at School: Educational Neuroscience in the Classroom. Open
University Press

Hebb, D.O. (1949), The organization of behaviour, New York: Wiley

Johnson, N. & Hogg, D. (1995) Learning the Distribution of Object Trajectories for Event
Recognition. Proc. British Machine Vision Conference, volume 2, pages 583-592 [accessed June
2010 at www.leeds.ac.uk]

Kelso, J.A.S (1995) Dynamic Patterns: The self-Organisation of Brain and Behaviour. Cambridge,
Massachusetts: MIT Press

Kuo, R. J. & Xue, K. C. (1999) Fuzzy neural networks with application to sales forecasting. Fuzzy
Sets and Systems 108:123-143

Sylwester, R. (1995). A celebration of neurons: an educator's guide to the brain. Tuscon, AZ:
Zephyr Press.

Page | 6