Beruflich Dokumente
Kultur Dokumente
2) YASER NOROUZI
Assistant Professor, Electrical Engineering Department,
Amirkabir University of Technology
Tehran, Iran
y.norouzi@aut.ac.ir
3) MARJAN FARIBORZ
B.S.c Student, Electrical Engineering Department,
Amirkabir University of Technology
Tehran, Iran
mfariborz@aut.ac.ir
ABASTARCT
In this paper, we present a feedforward training
algorithm using Regularized Logistic Regression and
Neural Networks to recognize handwritten objects.
Furthermore, we intend to consider the effect of
Gaussian noise in this procedure in order to examine the
versatility of our approach. We might intend to transmit
the image of our digits through an AWGN channel to a
certain destination and then do the recognition process
in our destination, so we need our algorithm to be still
robust against the noises caused by AWGN channels
and sensors. The main advantage of our approach is to
reduce the amount of computations and, in turn,
considerably decrease the processing time.
Index Terms Learning, Neural networks, AWGN,
Feedforward neural networks
1. INTRODUCTION
Regarding recent progress in processing technology,
lots of algorithms have emerged in order to classify and
recognize hand written objects, paving the way for the
idea of numerous useful applications finding a way to
recognize and determine handwritten notes has always
been an obsession for scientists. This can be so
beneficial in various applications in which the operators
tend to unify and integrate all the handwritten notes
with the aim of facilitating and enhancing the utilization
of the notes. Automated handwritten digit recognition is
widely used today from recognizing zip codes (postal
codes) on mail envelopes to recognizing the amounts
written on bank checks. In this paper we incline to
introduce an algorithm, called Logistic Regression, to
recognize handwritten notes and then we will amend it
by the use of neural networks. As a brief summary, in
MULTI-CLASS CLASSIF
FICAION
(x ( 2 ) ) T
(1)
X =
(m ) T
(x
)
2.2.
2.3.
1 m
[ y(i) Log(h (x (i) )) (11 y(i) ) Log(1 h (x (i) ))]
m i =1
(2)
g(
T (i)
x ) and g(z) =
-1
(x(1) )T
(2) T
(x )
X =
#
(m) T
(x )
0
1
and =
#
n
(3)
(x (1) ) T T (x (1) )
(x ( 2 ) ) T T (x ( 2 ) ) (4)
=
X =
#
#
(m) T
T
(m)
(x ) (x )
In the last equation, we used the fact that aTb = bTa if a
and b are vectors. This allows us to compute the
products T x(i) for all our examples i in one line of our
code. Our implementation uses the above mentioned
strategy to calculate T x(i). We also use a vectorized
approach for the rest of the cost function. A fully
vectorized version of logistic regression cost function
does not contain any loops.
2.3.2. Vectorizing the Gradient
Note that the gradient of the (unregularized) logistic
regression cost is a vector where the jth element is
defined as [16]:
1 m
1
((h (x(i) ) y(i) ) x(i) ) = X T (h (x) y)
m i =1
m
(6)
Where:
h (x (1) ) y(1)
h (x (2) ) y(2)
h (x) y =
(m)
(m)
h (x ) y
(7)
(8)
1 m
n 2
[ y(i) Log(h (x (i) )) (1 y(i) ) Log(1 h (x (i) ))] +
j
2m j =1
m i =1
(9)
(5)
J
m
(i)
(i)
(i)
= 1 ((h (x ) y ) x 1 )
1 m i =1
#
#
J
((h (x (i) ) y (i) ) x (i) )
n
i =1
|
| 1
|
J
1 m
= ((h (x (i) ) y (i) ) x (i)j )
j m i =1
J ( )
1
=(
0
m
(h(x
(i)
) y (i) ) x (i)j ) +
i =1
for j1
(11)
X n e w = X + n (t)
NEURAL NETWORK
KS
After adding noise
1 "
X = # %
1 "
Then we compute z = X
x (1)
#
x (m)
(13)
and next
n C as:
1
(14)
C=
1 + e z
90.5 %
86.8 %
5. CONCLUSION
4.
STATISTICAL ANALYSIS
REFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
http://sparkpublic.s3.amazonaws.com/ml/exercises/
ex2.zip
[22]
Friedman,
in multiple
the Royal
59, no. 1,