Beruflich Dokumente
Kultur Dokumente
Let k = 1;
while there exist input vectors that are misclassified by wk-1, do
Then the dividing line crosses at (0, 1)T and (-2, 0)T.
1 2 3
-1
-3 -2 -1
-1 -2 -3
i1
class -1 class 1
September 23, 2010
The new dividing line crosses at (0, 1)T and (-1/3, 0)T.
i2
3 2 1
-1
Let us pick the next misclassified point (0, 2)T for learning:
1 2 3
-3 -2 -1
-1 -2 -3
i1
class -1 class 1
September 23, 2010 Neural Networks Lecture 6: Perceptron Learning 4
3 2 1
1
1 2 3
With this weight vector, the perceptron achieves perfect classification! The learning process terminates.
-1
class -1 class 1
-1 -2 -3
i1
In most cases, many more iterations are necessary than in this example.
Neural Networks Lecture 6: Perceptron Learning 5
Adalines
Idea behind adaptive linear elements (Adalines): Compute a continuous, differentiable error function between net input and desired output (before applying threshold function). For example, compute the mean squared error (MSE) between every training vector and its class (1 or -1).
Gradient Descent
Gradient descent is a very common technique to find the absolute minimum of a function. It is especially useful for high-dimensional functions. We will use it to iteratively minimizes the networks (or neurons) error by finding the gradient of the error surface in weight-space and adjusting the weights in the opposite direction.
10
Gradient Descent
Gradient-descent example: Finding the absolute minimum of a one-dimensional error function f(x):
f(x) slope: f(x0)
x0
x1 = x0 - f(x0)
Repeat this iteratively until for some xi, f(xi) is sufficiently close to 0.
September 23, 2010 Neural Networks Lecture 6: Perceptron Learning 11
Gradient Descent
Gradients of two-dimensional functions:
The two-dimensional function in the left diagram is represented by contour lines in the right diagram, where arrows indicate the gradient of the function at different locations. Obviously, the gradient is always pointing in the direction of the steepest increase of the function. In order to find the functions minimum, we should always move against the gradient.
September 23, 2010 Neural Networks Lecture 6: Perceptron Learning 12