Neural Network

LeNet-5
 Object detection using R-CNN
The basic idea of R-CNN is to take a deep Neural Network
which was originally trained for image classification using
millions of annotated images and modify it for the purpose of
object detection. The basic idea from the first R-CNN paper is
illustrated in the Figure below (taken from the paper): (1)
Given an input image, (2) in a first step, a large number region
proposals are generated. (3) These region proposals, or
Regions-of-Interests (ROIs), are then each independently
sent through the network which outputs a vector of e.g. 4096
floating point values for each ROI. Finally, (4) a classifier is
learned which takes the 4096 float ROI representation as
input and outputs a label and confidence to each ROI.
Supervised vs Unsupervised
Learning

• Supervised Learning
– Data: (x, y) x is data, y is label
– Goal: Learn a function to map x -
>y
– Examples: Classification,
regression, object detection,
semantic segmentation, image
captioning, etc.
Formal neuron - perceptron

5
6
 How do we actually use an artificial
neuron?
• Feedforward network: The neurons in each layer feed their
output forward to the next layer until we get the final output from
the neural network.

• There can be any number of hidden layers within a feedforward

network.

• The number of neurons can be completely arbitrary.

7
Neural Network – Back propagation
Example

https://mattmazur.com/2015/03/17/a-step-by-step-
backpropagation-example/

8
Artificial Neuron Model

Implementation of AND
function
Let W1 = W2 = 1

X1 X2 X1W1 + X2W2 Y
0 0 0 0
0 1 1 0
1 0 1 0
1 1 2 1

If we make  = 2 (or any value >1 but <=2), we will get

correct results with a unit step activation function
Artificial Neuron Model

If we place the 4 points in a

two coordinate system (X1
and X2), we have drawn a line
from (2, 0) to (0, 2) in the
resulting plane

Any new data falling on the

left side of the line will give an
output of zero and the data on
the right side of the line will be
classified as one
Linearly Separable Problems

If the data can be correctly divided into two categories by a

line or hyper-plane
Non Linear Problems

If the data cannot be correctly separated by a single line or

plane, then it is not linearly separable. A single layer of
neurons cannot classify the input patterns that are not
linearly separable; e.g. exclusive OR problem
Non Linear Problems

For such problems we need two or more layers of neurons

-1 0.5
1 -1

x1 -1 -1
1

-0.5
x2
1

-1 -1
The input sum for the neuron of the 2nd layer = Y1 – Y2 + 1

For point (0,1) Y1 = 0, Y2 = 0, Sum = 1, Y3 = 0

For point (1,0) Y1 = 1, Y2 = 1, Sum = 1, Y3 = 0
For point (0,0) Y1 = 0, Y2 = 1, Sum = 0, Y3 = 1
For point (1,1) Y1 = 0, Y2 = 1, Sum = 0, Y3 = 1

Thus the neuron of the 2nd layer separates the data correctly