240-650 Principles of Pattern Recognition

Montri Karnjanadecha montri@coe.psu.ac.th http://fivedots.coe.psu.ac. th/~montri

240-572: Chapter 1: Introduction 1

Chapter 1
Introduction

240-572: Chapter 1: Introduction

Outline
Pattern Recognition System The Design Cycle Learning and Adaptation Read Chapter 1 (Duda, Hart, and Stork)

240-572: Chapter 1: Introduction

Motivations
Pattern recognition has many very valuable civil as well as military applications
Automated target recognition Automated Processing Systems New Human Computer Interface Biometrics

240-572: Chapter 1: Introduction

Handwritten Address Interpretation System

HWAI - http://www.cedar.buffalo.edu/HWAI/
The HWAI (Handwritten Address Interpretation) System was developed at Center of Excellence for Document Analysis and Recognition (CEDAR) at University at Buffalo, The State University of New York. It resulted from many years of research at CEDAR on the problems of Address Block location, Handwritten Digit/Character/Word Recognition, Database Compression, Information Retrieval, Real-Time Image Processing, and LooselyCoupled Multiprocessing.

The following presentation is based on the demonstration pages at HWAI

240-572: Chapter 1: Introduction

Handwritten Address Interpretation System

cont.

Step 1: Digitization

240-572: Chapter 1: Introduction

Handwritten Address Interpretation System

Cont.

Step 2: Address Block Location

240-572: Chapter 1: Introduction

Handwritten Address Interpretation System

Cont.

Step 3: Address Extraction

240-572: Chapter 1: Introduction

Handwritten Address Interpretation System

Cont.

Step 4: Binarization

240-572: Chapter 1: Introduction

Handwritten Address Interpretation System

Cont.

Step 5: Line Separation

240-572: Chapter 1: Introduction

10

Handwritten Address Interpretation System

Cont.

Step 6: Address Parsing

240-572: Chapter 1: Introduction

11

Handwritten Address Interpretation System

Cont.

Step 7: Recognition
(a) State Abbreviation Recognition

240-572: Chapter 1: Introduction

12

Handwritten Address Interpretation System

Cont.

Step 7: Recognition
(b) ZIP Code Recognition

240-572: Chapter 1: Introduction

13

Handwritten Address Interpretation System

Cont.

Step 7: Recognition
(c) Street Number Recognition

240-572: Chapter 1: Introduction

14

Handwritten Address Interpretation System

Cont.

Step 8: Street Name Recognition

240-572: Chapter 1: Introduction

15

Handwritten Address Interpretation System

Cont.

Step 9: Delivery Point Codes

240-572: Chapter 1: Introduction

16

Handwritten Address Interpretation System

Cont.

Step 10: Bar coding

240-572: Chapter 1: Introduction

17

IBM Voice Systems

Voice enabling e-bussiness
http://www-4.ibm.com/software/speech/enterprise/dcenter/demo_0.html

Get information through speech recognition software ViaVoice

240-572: Chapter 1: Introduction

18

Machine Demonstrates Superhuman Speech Recognition Abilities

Developed by Jim-Shih Liaw and Theodore W. Berger at University of Southern California The following is the claim
University of Southern California biomedical engineers have created the world's first machine system that can recognize spoken words better than humans can. A fundamental rethinking of a long-underperforming computer architecture led to their achievement .

240-572: Chapter 1: Introduction

19

240-572: Chapter 1: Introduction

20

Statistical Pattern Recognition

In statistical pattern recognition, recognition is done by classifying the input (represented as a set of measurements) into predefined categories The core questions we want to address
What is the best we can do (statistically) when a set of measurements is given for input? Which measurements should be used if we can choose a subset of all the measurements?

240-572: Chapter 1: Introduction

21

A Simple Example
Suppose that we are given two classes [1 and [2
P([1) = 0.7 P([2) = 0.3 No measurement is given
Guessing

What shall we do to recognize a given input? What is the best we can do statistically? Why?

240-572: Chapter 1: Introduction

22

An Introductory Example

240-572: Chapter 1: Introduction

23

Terminology
Features
Measurements available to the pattern recognition system

Models
Each class is represented by a description in mathematical forms, called a model

Preprocessing
Segmentation
Isolate the object of interest from the background and other objects

240-572: Chapter 1: Introduction

24

Terminology - cont.
Feature extraction
Is the measuring process that produces the measurements, or called features

Training samples
Models for classes are often specified by samples with known labels. These samples are called training samples

240-572: Chapter 1: Introduction

25

Terminology - cont.
Cost/risk
The cost of a decision associated with the recognition result

Decision theory
The theory on optimal decision rules

240-572: Chapter 1: Introduction

26

Terminology - cont.
Decision boundary
Boundaries in the feature space of regions with different classes (decisions)

240-572: Chapter 1: Introduction

27

Terminology - cont.
Generalization
While classes can be specified by training samples with known labels, the goal of a recognition system is to recognize novel inputs When a recognition system is over-fitted to training samples, it may give bad performance for typical inputs

240-572: Chapter 1: Introduction

28

Terminology - cont.
Generalization - continued

240-572: Chapter 1: Introduction

29

Terminology - cont.
Generalization - continued

240-572: Chapter 1: Introduction

30

Terminology - cont.
Generalization - continued

240-572: Chapter 1: Introduction

31

Terminology - cont.
- Analysis by synthesis model

240-572: Chapter 1: Introduction

32

Designing a Pattern Recognition System

240-572: Chapter 1: Introduction

33

Designing a Pattern Recognition System

240-572: Chapter 1: Introduction

34

Steps in a Pattern Recognition System Sensing

Measuring of features, such as a digital camera, or a microphone We assume the measurements are given

Segmentation and grouping

In the fish example, we have to isolate a fish from other fishes, other non-fish objects, or the background Segmentation/grouping is a very difficult problem

240-572: Chapter 1: Introduction

35

Steps in a Pattern Recognition System

cont.

Image segmentation is one of the most difficult problems in computer vision

Face detection, for example, can be viewed as a image segmentation problem

240-572: Chapter 1: Introduction

36

Steps in a Pattern Recognition System

cont

In speech recognition, the segmentation problem is called source separation

Mixed speech signal Separated signal source 1 Separated signal source 2

240-572: Chapter 1: Introduction

37

Steps in a Pattern Recognition System

cont.

Feature extraction/selection
A critical step for pattern recognition Seeking distinguishing features that are invariant to irrelevant transformations of the input Biometrics can be viewed as a feature selection problem

Classification Post-processing
Context information Multiple classifiers
240-572: Chapter 1: Introduction 38

The Design Cycle

240-572: Chapter 1: Introduction

39

Learning
Supervised learning
A category label is given for each pattern in a training set

Unsupervised learning
The system forms clusters or natural groupings of the input patterns The study of category formation

Reinforcement learning
No desired output is provided; the feedback is given

240-572: Chapter 1: Introduction

40