1

Contents:
 Multilayer Perceptron (MLP)
 Introduction to Back Propagation Algorithm
 Derivation of Back Propagation of Error
 Advantages of Back Propagation Algorithm
 Disadvantages of Back Propagation Algorithm
 Application of Back Propagation Algorithm
 2

BACK PROPAGATION ALGORITHM IN MULTILAYER PERCEPTRON

Multilayer Perceptron (MLP):

A multilayer perceptron is a
neural network connecting
multiple layers in a directed
graph, which means that the
signal path through the nodes
only goes one way. Each
node, apart from the input
nodes, has a nonlinear
activation function. An MLP
uses back propagation as a
supervised learning
technique.
 3

BACK PROPAGATION ALGORITHM IN MULTILAYER PERCEPTRON

Introduction to Back Propagation Algorithm:
Back-propagation is the essence of neural
network training. It is the method of fine-
tuning the weights of a neural net based on
the error rate obtained in the previous epoch
(i.e., iteration). Proper tuning of the weights
allows you to reduce error rates and to make
the model reliable by increasing its
generalization.
Back propagation is a short form for
"backward propagation of errors." It is a
standard method of training artificial neural
networks. This method helps to calculate the
gradient of a loss function with respects to all
the weights in the network.
 4

BACK PROPAGATION ALGORITHM IN MULTILAYER PERCEPTRON

Derivation of Back Propagation of Error:
 5

BACK PROPAGATION ALGORITHM IN MULTILAYER PERCEPTRON

The popularity of on-line learning for the supervised training of multilayer perceptron has
been further enhanced by the development of the back-propagation algorithm. To describe
this algorithm, consider the given figure on the previous page (fig 4.3), which depicts neuron j
being fed by a set of function signals produced by a layer of neurons to its left. The induced
local field vj(n) produced at the input of the activation function associated with neuron j is
therefore
 6

BACK PROPAGATION ALGORITHM IN MULTILAYER PERCEPTRON

In a manner similar to the Least Mean Square (LMS), the back
 7

BACK PROPAGATION ALGORITHM IN MULTILAYER PERCEPTRON

 8

BACK PROPAGATION ALGORITHM IN MULTILAYER PERCEPTRON

 9

BACK PROPAGATION ALGORITHM IN MULTILAYER PERCEPTRON

From Eqs. (4.15) and (4.16), we note that a key factor involved in the calculation of the weight
adjustment ∆wji(n) is the error signal ej(n) at the output of neuron j. In this context, we may
identify two distinct cases, depending on where in the network neuron j is located.
 10

BACK PROPAGATION ALGORITHM IN MULTILAYER PERCEPTRON

 11

BACK PROPAGATION ALGORITHM IN MULTILAYER PERCEPTRON

Fig 4.4: Signal-flow graph highlighting the details of output neuron k connected to hidden neuron j.
 12

BACK PROPAGATION ALGORITHM IN MULTILAYER PERCEPTRON

 13

BACK PROPAGATION ALGORITHM IN MULTILAYER PERCEPTRON

 14

BACK PROPAGATION ALGORITHM IN MULTILAYER PERCEPTRON

 15

BACK PROPAGATION ALGORITHM IN MULTILAYER PERCEPTRON

Advantages of Back Propagation Algorithm:

 It is fast, simple and easy to program.
 It has no parameters to tune (except for the number of input) .
 This is a shift in mind set for the learning-system designer instead of trying to design a
learning algorithm that is accurate over the entire space
 It requires no prior knowledge about the weak learner and so can be flexible.

Disadvantages of Back Propagation Algorithm:

 The actual performance of Back propagation on a particular problem is clearly dependent on
the input data.
 Back propagation can be sensitive to noisy data and outliers.
 Fully matrix-based approach to back propagation over a mini-batch .
 16

BACK PROPAGATION ALGORITHM IN MULTILAYER PERCEPTRON

Applications of Back Propagation Algorithm:

 Mapping character strings into phonemes so they can be pronounced by a computer.
 Neural network trained how to pronounce each letter in a word in a sentence, given the three
letters before and three letters after it in a window
 In the field of Speech Recognition.
 In the field of Character Recognition.
 In the field of Face Recognition.
 17

Thank You!