Sie sind auf Seite 1von 25

# Support Vector Machine

SVM
MATLAB code examples

Poli Amlo

Agosto 2014

Poli Amlo

Outline

Poli Amlo

Outline

Poli Amlo

## Support Vector Machine

Introduction
Support Vector Machine (SVM) is a tool that we can use to
classification between a sets. We have a set xi of training data
and labels yi that describe at which class belong.
Then we have data training xi and labels yi , now we want
to predict at which class belongs a new element x with the
similar characteristics of training data.
When you are working with classification is because you
have your elements as an arrange of vectors, if not we suggest that you transform your data in vectors, you can do it
using kernels. But if you are new using SVM we suggest to
do the simple case using points of two elements to plot them
in the Cartesian plane and it could help you to see clearest all
the steps for SVM.

Poli Amlo

## The simple case

The simplest case is the classification of two sets, in this
example we choose 6 points. . .
Training data
Class one x1 = (2, 2), x2 = (3, 1), x3 = (3, 4)
Class two x4 = (7, 2), x5 = (6, 4), x6 = (8, 4)

Labels
Class one y1 = 1, y2 = 1, y3 = 1
Class two y4 = 1, y5 = 1, y6 = 1

Poli Amlo

## The simple case

The simplest case is the classification of two sets, in this
example we choose 6 points. . .
Training data
Class one x1 = (2, 2), x2 = (3, 1), x3 = (3, 4)
Class two x4 = (7, 2), x5 = (6, 4), x6 = (8, 4)

Labels
Class one y1 = 1, y2 = 1, y3 = 1
Class two y4 = 1, y5 = 1, y6 = 1

Poli Amlo

## The simple case

The simplest case is the classification of two sets, in this
example we choose 6 points. . .
Training data
Class one x1 = (2, 2), x2 = (3, 1), x3 = (3, 4)
Class two x4 = (7, 2), x5 = (6, 4), x6 = (8, 4)

Labels
Class one y1 = 1, y2 = 1, y3 = 1
Class two y4 = 1, y5 = 1, y6 = 1

Poli Amlo

## The simple case

The simplest case is the classification of two sets, in this
example we choose 6 points. . .
Training data
Class one x1 = (2, 2), x2 = (3, 1), x3 = (3, 4)
Class two x4 = (7, 2), x5 = (6, 4), x6 = (8, 4)

Labels
Class one y1 = 1, y2 = 1, y3 = 1
Class two y4 = 1, y5 = 1, y6 = 1

Poli Amlo

MATLAB Code

## clc; clear all;close all;

X=zeros(6,2);
X(1,:)=[2,2];
X(2,:)=[3,1];
X(3,:)=[3,4];
X(4,:)=[7,2];
X(5,:)=[6,4];
X(6,:)=[8,4];
figure
for i=1:6
scatter(X(i,1), X(i,2)); hold on
end

Poli Amlo

C+ =

1
xi
m+

(1)

C =

1
xi
m

(2)

## Where m+ and m are the number of elements for the two

classes. In our example m+ = 3 and m = 3.
We calculated them and we obtained : C+ = (2.666, 2.333)
and C = (6.0, 3.5)

Poli Amlo

Plotting means

Poli Amlo

## Support Vector Machine

MATLAB Code

Cplus = mean(X(1:3,:));
Cminus = mean(X(3:6,:));
figure
scatter(Cplus(1,1), Cplus(1,2)); hold on
scatter(Cminus(1,1), Cminus(1,2));
axis([0 10 0 6]); grid on;

Poli Amlo

## Calculating the middle point

Now we need to obtain the middle point and it will be useful to predict new points.
(C+ + C )
2
In our case this point is : C = (4.333, 2.916)
C=

Poli Amlo

(3)

## Predicting new points

For new points x that we will want to classify we calculate
B = x C and W = C+ C . We verify the angle between
these vectors B and W and if the angle is less than 90 degrees then it belongs to the first class if not then it belong to
the second class.
For example, for the point x = (5, 2) we can see the plot
and guess where it belongs, with the algorithm we obtain that
it belongs to second class due to the angle is 106.73 degrees

Poli Amlo

## Predicting new points

Another example, x = (2, 2). For this new point we obtained
and angle of 2.15 degrees so this point belongs to the first
class.

Poli Amlo

## Support Vector Machine

MATLAB Code

x=[5,2];
scatter(x(1),x(2),green);hold on;
B=x-C;
W=Cplus-Cminus;
angle=dot(B,W)/(norm(B)*norm(W));
h=acos(angle);
a=h*180/pi

Poli Amlo

Labels

## Now we use the function sign to classify with the labels 1 if

it belongs to first class and 1 to second class.
y = sgn(h(x C), W i)

(4)

## For the point x = (5, 2) we obtain y = 1 and for the point

x = (2, 2) we obtain y = 1.
MATLAB code :
y=sgn(dot(B,W));

Poli Amlo

Outline

Poli Amlo

## Support Vector Machine

Kernels

As we said in the first part, most of the time our data needs
to be in the featuring space, this means that we need to transform our data to obtain vectors that we can manage like in the
example. Schlkopf defines a kernel as :
Kernel
Function that, given two patterns x and x 0 returns a real
number characterizing their similarity.
A kernel k is symmetric (i.e. k (x, x 0 ) = k (x 0 , x))

Poli Amlo

Outline

Poli Amlo

Some concepts

## Once that we understood the idea of a basic classification,

we continue with some concepts that are necessary before
the next example.
We will use hyperplanes to classify our classes and we can
find the optimal hyperplane using :
max min kx xi k|x H , hw, xi + b = 0, i = 1, ..., m

xH ,b

Poli Amlo

(5)

Margins

## For a hyperplane x H |hw, xilb = 0 a margin is :

hw,xi (x, y ) :=

y (hw, xi + b)
||w||

(6)

Note that when we multiply by y we are forcing that in the correct classification we will obtain a positive and almost if it is
correctly classify a point x the margin is the distance from x to
the hyperplane.
The margin will be 0 in the hyperplane. You can see the
geometrical representation in the next figure.

Poli Amlo

Margins

## F IGURE : Geometrical representation of margins 1 .

1. This image was taken from the book "Learning with Kernels", Schlkopf and Smola, 2002
Poli Amlo

Support Vectors

## The name of support vectors come from the optimization

problem to find the margins.

Poli Amlo