CHAPTER 2

ARTIFICIAL NEURAL NETWORKS

14

ARTIFICIAL NEURAL NETWORK

An artificial neuron network (ANN) is a computational model based on the structure and
functions of biological neural networks. A neural network is a massively parallel distributed
processor made of simple processing units called artificial neurons. Artificial neural networks
are very different from biological networks, although many of the concepts and characteristics of
biological systems are faithfully reproduced in the artificial systems. Artificial neural nets are a
type of non-linear processing system that is ideally suited for a wide range of tasks, especially
tasks where there is no existing algorithm for task completion. ANN can be trained to solve
certain problems using a teaching method and sample data. In this way, identically constructed
ANN can be used to perform different tasks depending on the training received. With proper
training, ANNs are capable of generalization, the ability to recognize similarities among different
input patterns, especially patterns that have been corrupted by noise. The artificial neural
network can be categorised into two parts namely neuron model and perceptron model.
2.1.1 ONE NEURON MODEL
In this model, neuron has several inputs and one output. Each input is associated with a
weight wi. The sum of product of inputs and respective weights is first calculated by the neuron.
This summation is the input for activation function which calculates the output.

Figure 2.1. Model of a neuron

15

2.1.2 PERCEPTRON MODEL

A more complex model called perceptron model (Kasabov et al. 1996) was later
introduced to extend the one neuron model. The concept of perceptron was the starting point for
the development of neural networks. A perceptron element consists of a single node which
receives weighted inputs and produces results using thresholds according to certain rules. The
perceptron is efficiently used to classify linearly separable data but is not suitable for nonlinear
data.
2.2 ARCHITECTURE OF NEURAL NETWORKS
A neural network consists of a number of interconnected nodes or neurons. Each node is
a simple processing element that responds to the weighted inputs it receives from the other
nodes. The arrangement of the nodes or neurons is referred as the network architecture. There are
a number of different parameters that must be decided upon when designing a neural network.
Among these parameters are the number of layers, the number of nodes or neurons per layer, the
number of training iterations, etc. Some of the more important parameters in terms of training
and network capacity are the number of hidden neurons, the learning rate and the momentum
parameter. The network can learn more complex problems by increasing the number of hidden
layers.
2.3 TRAINING OF NEURAL NETWORK
Once a network has been structured for a particular application, that network is ready to
be trained. To start this process the initial weights are chosen randomly. Then, the training, or
learning, begins. There are two approaches to training - supervised and unsupervised. Supervised
training involves a mechanism of providing the network with the desired output either by
manually "grading" the network's performance or by providing the desired outputs with the
inputs. Unsupervised training is where the network has to make sense of the inputs without
outside help.

16

2.3.1 SUPERVISED TRAINING

In supervised training, both the inputs and the outputs are provided. The network then
processes the inputs and compares its resulting outputs against the desired outputs. Errors are
then propagated back through the system, causing the system to adjust the weights which control
the network. This process occurs over and over as the weights are continually tweaked. The set
of data which enables the training is called the "training set." During the training of a network
the same set of data is processed many times as the connection weights are ever refined.
2.3.2 UNSUPERVISED TRAINING
The other type of training is called unsupervised training. In unsupervised training, the
network is provided with inputs but not with desired outputs. The system itself must then decide
what features it will use to group the input data. This is often referred to as self-organization or
adaption
2.4 FEED FORWARD BACK PROPAGATION ALGORITHM FOR TRAINING OF ANN
Since the real uniqueness or 'intelligence' of the network exists in the values of the
weights between neurons, we need a method of adjusting the weights to solve a particular
problem. For this type of network, the most common learning algorithm is called Back
Propagation (BP). A back propagation network learns by example, that is, we must provide a
learning set that consists of some input examples and the known-correct output for each case. So,
we use these input-output examples to show the network what type of behaviour is expected, and
the back propagation algorithm allows the network to adapt.
The back propagation learning process works in small iterative steps: one of the example
cases is applied to the network, and the network produces some output based on the current state
of its synaptic weights (initially, the output will be random). This output is compared to the
known-good output, and a mean-squared error signal is calculated. The error value is then
propagated backwards through the network, and small changes are made to the weights in each
layer. The weight changes are calculated to reduce the error signal for the case in question. The
whole process is repeated for each of the example cases, then back to the first case again, and so
on. The cycle is repeated until the overall error value drops below some pre-determined

17

threshold. The network will never exactly learn the ideal function, but rather it will
asymptotically approach the ideal function.
2.5 FACTORS AFFECTING THE PERFORMANCE OF THE NEURAL NETWORK
The factors affecting the performances of the neural network to generalize, that is, the
capability of the neural network to interpolate and extrapolate to data that it has not seen before
are as follows
2.5.1 NUMBER OF NODES
The mathematical structure can be made very flexible and the neural network can be used
for a wide range of applications by using large number of simple processing elements. However,
it may not be necessary for all applications because very simple topologies have been
investigated using a small number of data points. The neural network ability to represent the
training data increases with the number of nodes in the hidden layer(s) at the expense of its
capability to generalize.
2.5.2 SIZE OF TRAINING DATA SET
The data set used must be representative of the entire distribution of values corresponding
to particular class. If the entire set of the distribution of the data in feature space is not covered
sufficiently the network may not classify new data accurately. Therefore sufficient number of
data is often required for training, and researchers are often concerned with finding the minimum
size of data set necessary. However, large training data sets also need longer training times.
Several modifications to the MLP algorithm have been introduced including the momentum
term, the delta-bar-delta rule, and optimization procedures to speed up the training process.
2.5.3 TRAINING TIME
The generalizing capability of the network is affected by the time taken for training a
network. The network trained on a specific data set for longer time may classify those data more
accurately; however, the capability to classify previously unseen data decreases consequently.
Therefore, an over trained network is able to memorize the training data but fails to generalize
when it is applied to different data sets. The accuracy of a trained neural network is evaluated
with a testing data set. The testing data set allows anyone to assess the accuracy of the trained
neural network. The choice of data for testing should, like the training data, be representative of
18

the entire distribution of values corresponding to a given class. The available data is generally
divided into training and testing data set and the test data is used to evaluate the performance of
the model.
2.6 ARTIFICIAL NEURAL NETWORKS IN REMOTE SENSING:
Artificial neural networks (ANN) have become an important tool in the analysis of
remotely sensed data mainly due to their ability to learn complex relationships. It has been
shown from the research that ANNs may be used to classify remotely sensed data more
accurately (Gopal et al. 1999; Liu et al. 2003; Murthy et al. 2003). Since artificial neural
networks are capable of adjusting its synaptic weights to adapt to the environment, it is able to
deal with incomplete information and provide responses under uncertainty.
Artificial neural networks can be used for surface water quality assessment (Gross et al.
1999), retrieval of soil moisture (Chang et al. 2000, Del Frate et al. 2003), estimation of crop
variables, predicting the yield of a crop, classification of different crops etc.
Artificial neural networks are also used to detect the disease in plants. Aji et al. (2013)
applied an ANN to detect pathogens which decrease oil production. Xiaoli Li and Yong He
(2008) applied an ANN in their study on tea leaves. They were able to discriminate the low
quality tea leaves and obtained a good accuracy of 77.3% in classification of all three tea gardens
by using ANN models. ANNs have also been used for the identification of plant viruses. The
results obtained indicated that the method using ANNs can be a reliable tool, very helpful in such
analyses. Therefore, it was suggested to use ANN models as an alternative for traditional
methods used in verification of a large amount of data (Glezakos et al. 2010).
Artificial neural networks are widely used in the field of agriculture, such as
classification of hard red wheat by feed forward back propagating neural networks (Chen et al.
1995), tomato maturity evaluation using colour image analysis (Choi et al. 1995), single wheat
kernel color classification (Wang 1999) retrieval of crop parameters of spinach (pandey et al.
2010) and estimation of rice crop variables (D.K. Gupta et al. 2015). Nakano et al.(1992)
developed a method that can classify the quality of the external appearance of apples using a
neural network. It can be used to evaluate the quality of the apples.

19