Classification using Neural Network

The objective is to write a code for classification of data using neural network. We will use
iris dataset for the same. Iris is a plant that has three different species, Versicolor, Setosa
and Virginica. Each of these species have a characteristic length and width of sepal and
petal.

Image Source: https://s3.amazonaws.com/assets.datacamp.com/blog_assets/Machine+Learning+R/iris-

machinelearning.png

We will use backpropagation algorithm to train on 85% (think why?) of the dataset. The
algorithm uses parameters like length of sepal, width of sepal, length of petal and width of
petal to train a neural network to classify iris plant into three different species; setosa,
virginica and versicolor.
We will then test the trained neural network on remaining 15% of the iris dataset .
Let us first import the dataset of iris.
data(iris)
head(iris)

## Sepal.Length Sepal.Width Petal.Length Petal.Width Species

## 1 5.1 3.5 1.4 0.2 setosa
## 2 4.9 3.0 1.4 0.2 setosa
## 3 4.7 3.2 1.3 0.2 setosa
## 4 4.6 3.1 1.5 0.2 setosa
## 5 5.0 3.6 1.4 0.2 setosa
## 6 5.4 3.9 1.7 0.4 setosa

Let us first explore the data by plotting Sepal Length-Width and Petal Length-Width. We
will use ggplot function of R that can be used for sophisticated data visualization. For using
ggplot function, we will use the library ggplot2.
library(ggplot2)
#plot Sepal Length-Width
scatter <- ggplot(data=iris, aes(x = Sepal.Length, y = Sepal.Width))
scatter + geom_point(aes(color=Species, shape=Species)) +
xlab("Sepal Length") + ylab("Sepal Width") +
ggtitle("Sepal Length-Width")

#plot Petal Length-Width

scatter <- ggplot(data=iris, aes(x = Petal.Length, y = Petal.Width))
scatter + geom_point(aes(color=Species, shape=Species)) +
xlab("Petal Length") + ylab("Petal Width") +
ggtitle("Petal Length-Width")

Now, let’s get started with training of neural network on the iris dataset!
We will use the sample function to randomly sample (why sample randomly?) around 85%
of the data for training and store the corresponding row index in index.
index <- sample(1:nrow(iris),round(0.85*nrow(iris)))

All the columns of iris dataset corresponding to our sampled data, index will be stored in
train_dataset and the remaining will be stored in test_dataset.
train_dataset <- iris[index,]
test_dataset <- iris[-index,]

Training of Neural Network for Classification

Now, we will import the neuralnet package.
library(neuralnet)

train_dataset$setosa = train_dataset$Species == "setosa"

train_dataset$virginica = train_dataset$Species == "virginica"
train_dataset$versicolor = train_dataset$Species == "versicolor"

For Species equal to setosa/virginica/versicolor, in the train_dataset, we generate 3 new

columns in the train_dataset dataframe corresponding to the three species that is set to be
TRUE whenever the species of train_dataset is the same as the new column. For example,
whenever Species in train_dataset is setosa, then only under the new column, setosa the
corresponding value is set to be TRUE and for the new columns, virginica and versicolor, it
is FALSE.
neuralnet is the function that is used to train neural network using backpropagation.
Hidden is a vector of integers that specifies number of neurons in each hidden layer.
Linear.output is set to be “False” for classification and “True” for Regression.
Backpropagation algorithm is used to calculate the weights.

model = neuralnet(versicolor + virginica + setosa~ Sepal.Length + Sepal.Width

+ Petal.Length + Petal.Width, train, hidden=c(4,2),linear.output = FALSE)

We will now see the results of our trained model. This includes the error, the number of
steps, the weights and bias terms of each of the layers of our neural network in a matrix
form.
A visual representation will follow later!

model$result.matrix

error 0.0007847987535
reached.threshold 0.0098530835824
steps 4774.0000000000000
Intercept.to.1layhid1 0.9597163710546
Sepal.Length.to.1layhid1 -0.6220062099427
Sepal.Width.to.1layhid1 3.0707032675327
Petal.Length.to.1layhid1 -1.8092003579257
Petal.Width.to.1layhid1 -0.1204848497516
Intercept.to.1layhid2 -4.4257841778010
Sepal.Length.to.1layhid2 0.2142440708761
Sepal.Width.to.1layhid2 -1.0802141217306
Petal.Length.to.1layhid2 1.2439680991077
Petal.Width.to.1layhid2 -0.0986983732654
Intercept.to.1layhid3 -1.8911927196853
Sepal.Length.to.1layhid3 -0.8388365462509
Sepal.Width.to.1layhid3 -1.5299584974311
Petal.Length.to.1layhid3 1.2343415553666
Petal.Width.to.1layhid3 2.4316788540469
Intercept.to.1layhid4 0.5496530682487
Sepal.Length.to.1layhid4 0.5365838769002
Sepal.Width.to.1layhid4 -0.6655978747541
Petal.Length.to.1layhid4 -0.0341342644253
Petal.Width.to.1layhid4 -1.2557192696984
Intercept.to.2layhid1 114.0135668418742
1layhid.1.to.2layhid1 -191.0246330119312
1layhid.2.to.2layhid1 214.3868095208997
1layhid.3.to.2layhid1 221.1865807952976
1layhid.4.to.2layhid1 -18.5468202325508
Intercept.to.2layhid2 -1.7380386230254
1layhid.1.to.2layhid2 12.1597732967249
1layhid.2.to.2layhid2 -13.2225327557032
1layhid.3.to.2layhid2 -3.1556794566051
1layhid.4.to.2layhid2 17.8925475787581
Intercept.to.versicolor -73.8227085862938
2layhid.1.to.versicolor 36.1612463389437
2layhid.2.to.versicolor 67.2092578817426
Intercept.to.virginica 39.9086611952630
2layhid.1.to.virginica 25.9785369035462
2layhid.2.to.virginica -117.5030755440037
Intercept.to.setosa 6.0114543239557
2layhid.1.to.setosa -116.4319709045455
2layhid.2.to.setosa 30.7253463332087

The rows highlighted above represents the bias of final layer of the model. Similarly, all
other parameters of the above table can be seen in our plot in the next step.
We will now create a visual representation of the trained model that labels the weights (in
black) and bias (in blue) explicitly in the model.
plot(model)
Testing of Trained Neural Network
Let us now test our model!
For this, we first subset the test_dataset we created earlier selecting the parameters
Sepal.Length, Sepal.Width, Petal.Length and Petal.Width and store it in temp_test.
temp_test <- subset(test_dataset,select = c("Sepal.Length","Sepal.Width","Pet
al.Length","Petal.Width"))

The compute function creates the prediction model for the trained neural network.

Each row of the results$net.result represents the probability of the given observation to be
versicolor, virginica or setosa. The first column represents versicolor, the second column
represents virginica and the third column represents setosa. The order of the columns is in
the same order we used to train the neural network.
results <- compute(model,temp_test)
results$net.result

## [,1] [,2] [,3]

## 7 0.005656983988 0.01654714974 0.993997577135
## 8 0.005653958927 0.01658496556 0.994107186702
## 12 0.005661790320 0.01649491575 0.993837792636
## 13 0.005657646486 0.01653803277 0.993971314444
## 16 0.005641492234 0.01673695432 0.994528383589
## 18 0.005651084870 0.01661714449 0.994200878552
## 25 0.005699552137 0.01609394338 0.992439101259
## 29 0.005649005606 0.01664305329 0.994273497769
## 32 0.005668193256 0.01640349858 0.993562797844
## 36 0.005647964403 0.01665329502 0.994303462805
## 40 0.005653400456 0.01659118034 0.994125432650
## 42 0.005876861967 0.01428687945 0.980002095696
## 49 0.005646199829 0.01667961431 0.994373414356
## 61 0.516317685129 0.48350579355 0.004136712149
## 69 0.516328594097 0.48351326840 0.004136253180
## 70 0.516321303571 0.48350844111 0.004136577614
## 116 0.516328990083 0.48351354718 0.004136237305
## 132 0.516329006062 0.48351356006 0.004136236836
## 138 0.516328967098 0.48351353437 0.004136238582
## 142 0.516328959623 0.48351352462 0.004136238409
## 143 0.516328969861 0.48351353367 0.004136238193
## 148 0.516328961380 0.48351352735 0.004136238495

Based on the above results, we create a matrix of our predictions.

preds <- factor(c( "versicolor", "virginica","setosa"))[apply(results$net.res
ult, MARGIN=1, FUN=which.max)]
table(preds, test_dataset$Species)

##
## preds setosa versicolor virginica
## setosa 7 0 0
## versicolor 0 8 0
## virginica 0 1 6

We observe that all setosa species of iris plant were accurately predicted. There was 1 false
prediction for versicolor.