LEARNING PARADIGMS

Syed Ali
Syed Ali Raza
Raza
Lecturer
Lecturer
GC University
GC University Lahore
Lahore
WHY “LEARN” ?
o Machine learning is to program computers to
optimize a performance criterion using example
data or past experience.
o Learning is used when:
o Humans are unable to explain their expertise (speech
recognition)
o Solution changes in time (routing on a computer
network)
o Solution needs to be adapted to particular cases (user
biometrics)
2
WHAT WE TALK ABOUT WHEN WE TALK
ABOUT“LEARNING”

o Given
o a data set D,
o a task T, and
o a performance measure M,
o A computer system is said to learn from D to
perform the task T if after learning the system’s
performance on T improves as measured by M.
o In other words, the learned model helps the system
to perform T better as compared to no learning.
A CLASSIC EXAMPLE
AN EXAMPLE APPLICATION
o An emergency room in a hospital measures n number of
variables (e.g., blood pressure, age, etc) of newly admitted
patients.
o A decision is needed: whether to put a new patient in an
intensive-care unit.
o Due to the high cost of ICU, those patients who may
survive less than a month are given higher priority.
o Problem: to predict high-risk patients and discriminate
them from low-risk patients.
ANOTHER EXAMPLE
o A credit card company receives thousands of applications for
new cards. Each application contains information about an
applicant
o Age
o Marital status
o Annual salary
o Outstanding debts
o Credit rating
o Problem: to decide whether an application should approved,
or to classify applications into two categories, approved and
not approved.
SOME MORE EXAMPLES

 Recognizing patterns:
 Facial identities or facial expressions
 Handwritten or spoken words
 Medical images
 Generating patterns:
 Generating images or motion sequences
 Recognizing anomalies:
 Unusual patterns of sensor readings in a nuclear power
plant or unusual sound in your car engine.
 Prediction:
 Future stock prices or currency exchange rates
SOME WEB-BASED EXAMPLES

 The web contains a lot of data. Tasks with very big datasets
often use machine learning
 especially if the data is noisy or non-stationary.

 Spam filtering, fraud detection:

 The enemy adapts so we must adapt too.

 Recommendation systems:
 Lots of noisy data. Million dollar prize!
 Information retrieval:
 Find documents or images with similar content.
 Data Visualization:
 Display a huge database in a revealing way
LEARNING ALGORITHM
o Learning: Learn a model using the training data
o Testing: Test the model using unseen test data to
assess the model accuracy

Number of correct classifications

Accuracy 
Total number of test cases

Training Learning
Learning Test
Training model
model Test Accuracy
Accuracy
Data Algorithm
Algorithm Data
Data Data

Step 1: Training Step 2: Testing

LEARNING PARADIGMS
o Supervised Learning
o Classification
o Regression
o Unsupervised Learning
o Clustering
o Reinforcement Learning
o Associative RL
o Non Associative RL

10
SUPERVISED LEARNING
CLASSIFICATION
o Predicts categorical class labels
o Classifies data based on the training set and the
values (class labels) in a classifying attribute and
uses it in classifying new data
o Credit scoring
o Differentiating between
low-risk and high-risk
customers from their
income and savings
CLASSIFICATION PROCESS

Classification
Classification
Algorithms
Algorithms

Training
Training Data
Data

Classifier
Classifier
(Model)
(Model)

IF
IF rank
rank =
= ‘professor’
‘professor’
OR
OR years
years >
> 66
THEN
THEN tenured
tenured =
= ‘yes’
‘yes’
CLASSIFICATION PROCESS:

Classifier
Classifier

Unseen
Unseen Data
Data
Testing
Testing Data
Data
(Jafri,
(Jafri, Professor,
Professor, 4)
4)

Tenured?
CLASSIFICATION: APPLICATIONS
o Pattern recognition
o Face detection and recognition
o Character recognition
o Speech recognition
o Use of a dictionary or the syntax of the language.
o Sensor fusion: Combine multiple modalities; eg,
visual (lip image) and acoustic for speech
o Medical diagnosis

15
REGRESSION
o Example: Price of a used car
o x : car attributes
y : price
y = g (x | θ )
g ( ) model,
θ parameters
REGRESSION APPLICATIONS
o Navigating a car: Angle of the steering wheel
o Kinematics of a robot arm

(x,y) α1= g1(x,y)

α2= g2(x,y)
α2

α1

 Response surface design

SUPERVISED LEARNING
o Prediction of future cases
o Use the rule to predict the output for future inputs
o Knowledge extraction
o Learning a rule from data
o Compression
o Finding a rule simpler than the data it explains
o Outlier detection
o Exceptions that are not covered by the rule, e.g., fraud

18
UNSUPERVISED LEARNING
UNSUPERVISED LEARNING
o Learning “what normally happens”
o No predefined output
o Clustering: Grouping similar instances
o Example applications
o Customer segmentation in CRM
o Image compression: Color quantization
o Bioinformatics: Learning motifs

20
CLUSTERING
o Clustering is a technique for finding similarity groups in
data, called clusters. I.e.,
o it groups data instances that are similar to (near) each other in one
cluster and data instances that are very different (far away) from
each other into different clusters.
o Clustering is unsupervised learning task as no class
values denoting an a priori grouping of the data instances
are given, which is the case in supervised learning.

21
AN ILLUSTRATION
o The data set has three natural groups of data
points, i.e., 3 natural clusters.
WHAT IS CLUSTERING FOR?
o Let us see some real-life examples
o Example 1: groups people of similar sizes
together to make “small”, “medium” and
“large” T-Shirts.
o Tailor-made for each person: too expensive
o One-size-fits-all: does not fit all.
o Example 2: In marketing, segment customers
according to their similarities
o To do targeted marketing.

23
WHAT IS CLUSTERING FOR? (CONT…)
o Example 3: Given a collection of text documents, we
want to organize them according to their content
similarities,
o To produce a topic hierarchy
o In fact, clustering is one of the most utilized data
mining techniques.
o It has a long history, and used in almost every field, e.g.,
medicine, psychology, botany, sociology, biology,
archeology, marketing, insurance, libraries, etc.
o In recent years, due to the rapid increase of online
documents, text clustering becomes important.
ASPECTS OF CLUSTERING
o A clustering algorithm
o Partitioning clustering
o Hierarchical clustering
o A distance (similarity, or dissimilarity) function
o Clustering quality
o The quality of a clustering result depends on the
algorithm, the distance function, and the application.
REINFORCEMENT LEARNING
o Reinforcement learning is supervised learning in which
limited information of the desired outputs is known
o Complete knowledge of the environment is not available;
only basic benefit or reward information
o In other words, a critic rather than a teacher guides the
learning process
o Reinforcement learning has roots in experimental
studies of animal learning
o Training a dog by positive (“good dog”, something to eat)
and negative (“bad dog”, nothing to eat) reinforcement

26
REINFORCEMENT LEARNING
o Associative
o Associating action and stimuli. In other words,
developing a action-stimuli mapping from reinforcement
information received from the environment.
o Non-associative
o Selecting one action instead of associating actions with
stimuli. The only input received from the environment is
reinforcement information. Examples include genetic
algorithms

27
 Arrows indicate
strength between
Start S2 two problem states

Start maze …

S4 S3

S8 S7

S5 Goal
 The first response leads

Start S2
to S2 …

The next state is

chosen by randomly
sampling from the
possible next states
S4 S3 weighted by their
associative strength

Associative strength =
line width
S8 S7

S5 Goal
 Suppose the randomly
Start S2 sampled response
leads to S3 …

S4 S3

S8 S7

S5 Goal
 At S3, choices lead to

Start S2
either S2, S4, or S7.

S7 was picked
(randomly)

S4 S3

S8 S7

S5 Goal
 By chance, S3 was

Start S2
picked next…

S4 S3

S8 S7

S5 Goal
 Next response is S4

Start S2

S4 S3

S8 S7

S5 Goal
 And S5 was chosen

Start S2
next (randomly)

S4 S3

S8 S7

S5 Goal
 And the goal is

Start S2
reached …

S4 S3

S8 S7

S5 Goal
 Goal is reached,

Start S2
strengthen the
associative connection
between goal state and
last response

Next time S5 is
S4 S3 reached, part of the
associative strength is
passed back to S4...

S8 S7

S5 Goal
 Start maze again…

Start S2

S4 S3

S8 S7

S5 Goal
 Let’s suppose after a

Start S2
couple of moves, we
end up at S5 again

S4 S3

S8 S7

S5 Goal
 S5 is likely to lead to

Start S2
GOAL through
strenghtened route

In reinforcement
learning, strength is
also passed back to the
S4 S3 last state

This paves the way for

the next time going
through maze
S8 S7

S5 Goal
 The situation after lots

Start S2
of restarts …

S4 S3

S8 S7

S5 Goal
KEY OBSERVATIONS
o The reinforcement signal can be any signal evaluating
the learning system's actions, not just a success/failure signal
o Often it takes on real values, and the objective of learning is
to maximize its expected value.
o The critic does not directly tell the learning system how to
change its actions.
o Reinforcement learning algorithms are selection processes.
There must be variety in the action-generation process so that
the consequences of alternative actions can be compared to
select the best.