Introduction to Machine Learning

Chapter 4 Introduction to
Machine Learning
Dr. Mohamad Hafis Izran Bin Ishak
Department of Control & Mechatronics (CMED)
Faculty of Electrical Engineering
Universiti Teknologi Malaysia.
SKEM 4173 Artificial Intelligence

www.utm.my
Dr. Hafis
Herman SKEM4173
innovative innovative
entrepreneurial
MKEM 1713 entrepreneurial global
global
Chapter Outline
Introduction to Machine Learning

Definition of Machine Learning
Types of Learning Algorithm
Examples of Learning Techniques
Decision tree
K-Nearest Neighbors
Linear regression
Logistic regression
Dr. Hafis
Herman SKEM4173
MKEM 1713 innovative entrepreneurial global
Introduction to Machine
Learning
Definition of Machine Learning
Examples of Learning Techniques
Dr. Hafis
Herman SKEM4173
MKEM 1713 innovative entrepreneurial global 3
Machine Learning is
Machine learning, a branch of artificial intelligence, concerns

the construction and study of systems that can learn from data.
Dr. Hafis
Herman SKEM4173
Machine Learning is
Machine learning is programming computers to optimize a

performance criterion using example data or past experience.
-- Ethem Alpaydin
The goal of machine learning is to develop methods that can

automatically detect patterns in data, and then to use the
uncovered patterns to predict future data or other outcomes of
interest.
-- Kevin P. Murphy
The field of pattern recognition is concerned with the automatic

discovery of regularities in data through the use of computer
algorithms and with the use of these regularities to take actions.
-- Christopher M. Bishop
Dr. Hafis
Herman SKEM4173
Machine Learning is
Machine learning is about predicting the future, based on the

past.
-- Hal Daume III
past future
Training model/ Testing model/

Data predictor Data predictor
Dr. Hafis
Herman SKEM4173
Show Time
Machine Learning (ML)
Introduction
What is ML?
Dr. Hafis
Herman SKEM4173
Algorithms
The success of machine learning system depends on the

algorithms.
The algorithms control the search to find and build the

knowledge structures.
The learning algorithms should extract useful information

from training examples.
Dr. Hafis
Herman SKEM4173
Supervised (inductive) learning

Training data includes desired outputs
Unsupervised learning
Training data does not include desired outputs
Semi-supervised learning
Training data includes a few desired outputs
Reinforcement learning
Rewards from sequence of actions
Dr. Hafis
Herman SKEM4173
Algorithms
Supervised learning Unsupervised learning
Semi-supervised learning
10
Dr. Hafis
Herman SKEM4173
Show Time
Supervised Learning
Dr. Hafis
Herman SKEM4173
Learning techniques
- Supervised Learning
Supervised learning categories and techniques:
Linear classifier (numerical functions)

Parametric (Probabilistic functions)
Nave Bayes, Gaussian discriminant analysis (GDA), Hidden Markov
models (HMM), Probabilistic graphical models
Non-parametric (Instance-based functions)
K-nearest neighbors, Kernel regression, Kernel density estimation,
Local regression
Non-metric (Symbolic functions)
Classification and regression tree (CART), decision tree
Aggregation
Bagging (bootstrap + aggregation), Adaboost, Random forest
Dr. Hafis
Herman SKEM4173
Learning techniques
- Supervised Learning
Linear classifier
Techniques:
Perceptron
Logistic regression
Support vector machine (SVM)
Ada-line
Multi-layer perceptron (MLP)
Dr. Hafis
Herman SKEM4173
Data
examples
Data
Dr. Hafis
Herman SKEM4173
Supervised learning
examples
label
label1
label2
labeled examples
label3
label4
Supervised learning: given labeled examples

Dr. Hafis
Herman SKEM4173
Supervised learning
label
label1
model/
label2 predictor
label3
label4
Supervised learning: given labeled examples
Dr. Hafis
Herman SKEM4173
Supervised learning
model/ predicted label

predictor
Supervised learning: learn to predict new example
Dr. Hafis
Herman SKEM4173
Classification Example
Differentiate
between low-risk
and high-risk
customers from
their income and
savings
Discriminant: IF income > 1 AND savings > 2

THEN low-risk ELSE high-risk
Dr. Hafis
Herman SKEM4173
Classification Applications
Face recognition
Character recognition
Spam detection
Medical diagnosis: From symptoms to illnesses
Biometrics: Recognition/authentication using physical

and/or behavioral characteristics: Face, iris, signature, etc
...
Dr. Hafis
Herman SKEM4173
Face Recognition
Training examples of a person
Test images
AT&T Laboratories, Cambridge UK

http://www.uk.research.att.com/facedatabase.html
Dr. Hafis
Herman SKEM4173
Regression Example
Price of a used car
x : car attributes y = wx+w0

(e.g. mileage)
y : price
Dr. Hafis
Herman SKEM4173
Regression Applications
Economics/Finance: predict the value of a stock
Epidemiology
Car/plane navigation: angle of the steering wheel,

acceleration,
Temporal trends: weather over time
Dr. Hafis
Herman SKEM4173
Show Time
Unsupervised Learning
Dr. Hafis
Herman SKEM4173
Learning techniques
- Unsupervised Learning
Unsupervised learning categories and techniques
Clustering
K-means clustering
Spectral clustering
Density Estimation
Gaussian mixture model (GMM)
Graphical models
Dimensionality reduction
Principal component analysis (PCA)
Factor analysis
Dr. Hafis
Herman SKEM4173
Unupervised learning: given data, i.e. examples, but no labels

Dr. Hafis
Herman SKEM4173
applications
learn clusters/groups without any label
customer segmentation (i.e. grouping)
image compression
bioinformatics: learn motifs

Dr. Hafis
Herman SKEM4173
Learning techniques
- Reinforcement Learning
left, right, straight, left, left, left, straight GOOD
left, straight, straight, left, right, straight, straight BAD
left, right, straight, left, left, left, straight 18.5
left, straight, straight, left, right, straight, straight -3
Given a sequence of examples/states and a reward after

completing that sequence, learn to predict the action to take
in for an individual example/state
Dr. Hafis
Herman SKEM4173
Reinforcement learning
example
Backgammon
WIN!
LOSE!
Given sequences of moves and whether or not the

player won at the end, learn to make good moves
Dr. Hafis
Herman SKEM4173
Other learning variations
What data is available:

Supervised, unsupervised, reinforcement learning
semi-supervised, active learning,
How are we getting the data:

online vs. offline learning
Type of model:
generative vs. discriminative
parametric vs. non-parametric
Dr. Hafis
Herman SKEM4173
Homework (2 marks each)
Please find more details about:

1. Decision tree
2. K-Nearest Neighbors
3. Linear regression
4. Logistic regression
What is it?
Advantages & Disadvantages
Example Applications
Dr. Hafis
Herman SKEM4173

Introduction to Machine Learning

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Introduction to Machine Learning

Hochgeladen von

Copyright:

Verfügbare Formate

Chapter 4 Introduction to

SKEM 4173 Artificial Intelligence

Introduction to Machine Learning

Machine learning, a branch of artificial intelligence, concerns

Machine learning is programming computers to optimize a

The goal of machine learning is to develop methods that can

The field of pattern recognition is concerned with the automatic

Machine learning is about predicting the future, based on the

Training model/ Testing model/

The success of machine learning system depends on the

The algorithms control the search to find and build the

The learning algorithms should extract useful information

Supervised (inductive) learning

Supervised learning Unsupervised learning

Linear classifier (numerical functions)

Supervised learning: given labeled examples

Supervised learning: given labeled examples

model/ predicted label

Supervised learning: learn to predict new example

Discriminant: IF income > 1 AND savings > 2

Medical diagnosis: From symptoms to illnesses

Biometrics: Recognition/authentication using physical

Training examples of a person

AT&T Laboratories, Cambridge UK

Price of a used car

x : car attributes y = wx+w0

Economics/Finance: predict the value of a stock

Car/plane navigation: angle of the steering wheel,

Temporal trends: weather over time

Unupervised learning: given data, i.e. examples, but no labels

customer segmentation (i.e. grouping)

bioinformatics: learn motifs

left, right, straight, left, left, left, straight GOOD

left, straight, straight, left, right, straight, straight BAD

left, right, straight, left, left, left, straight 18.5

left, straight, straight, left, right, straight, straight -3

Given a sequence of examples/states and a reward after

Given sequences of moves and whether or not the

What data is available:

How are we getting the data:

Please find more details about:

Das könnte Ihnen auch gefallen