22 views

Original Title: Instabce Based Learning

Uploaded by anandh_cdm

- Intro Sockets c
- Multi Class SVM - File Exchange - MATLAB Central
- Terrorist Network Analysis and Identification of Main Actors Using Machine Learning Techniques
- syllabus_EECS940_S12
- Class 01
- 5244338
- paper ID-28201441
- Predicting New Collaborations in Academic Citation Networks of IEEE and ACM Conferences
- Machine Learning Basic 2
- UCI Machine Learning Repository_ Abalone Data Set
- TermProj FR
- IRJET-Classification of Liver Disease Based on US Images
- Smart Camera
- SCL
- Design and Development of Forest Fire Management System
- CON 02
- Splunk Lisa2014 Final Version
- lesson plan 4
- Start.docx
- Vol21Issue1_2

You are on page 1of 16

Read Ch. 8] -Nearest Neighbor Locally weighted regression Radial basis functions Case-based reasoning Lazy and eager learning

k

199

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

Instance-Based Learning

Key idea: just store all training examples h ( )i Nearest neighbor: Given query instance , rst locate nearest training example , then estimate ^( ) ( ) -Nearest neighbor: Given , take vote among its nearest nbrs (if discrete-valued target function) take mean of values of nearest nbrs (if real-valued) P ( ) ^( )

x i ; f xi xq xn f xq f xn k xq k f k

k

f xq

i=1

f xi

200

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

Instances map to points in < Less than 20 attributes per instance Lots of training data Advantages: Training is very fast Learn complex target functions Don't lose information Disadvantages: Slow at query time Easily fooled by irrelevant attributes

n

201

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

Voronoi Diagram

+ xq + + +

202

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

Consider ( ) de nes probability that instance will be labeled 1 (positive) versus 0 (negative). Nearest neighbor: As number of training examples ! 1, approaches Gibbs Algorithm Gibbs: with probability ( ) predict 1, else 0

p x p x k k x

-Nearest neighbor: As number of training examples ! 1 and gets large, approaches Bayes optimal Bayes optimal: if ( ) 5 then predict 1, else 0

p x > :

Note Gibbs has at most twice the expected error of Bayes optimal

203

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

Distance-Weighted NN

k

k

f xq

i=1

wi f x i wi

where

wi

k i=1

and (

d xq ; xi

( ) ) is distance between

d xq ; xi k

xq

and

xi

Note now it makes sense to use all training examples instead of just ! Shepard's method

204

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

Curse of Dimensionality

Imagine instances described by 20 attributes, but only 2 are relevant to target function Curse of dimensionality: nearest nbr is easily mislead when high-dimensional One approach: Stretch th axis by weight , where chosen to minimize prediction error Use cross-validation to automatically choose weights Note setting to zero eliminates this dimension altogether

X j zj z1 ; : : : ; z n z1 ; : : : ; z n zj

205

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

Note NN forms local approximation to for each query point Why not form an explicit approximation ^( ) for region surrounding Fit linear function to nearest neighbors Fit quadratic, ... Produces \piecewise approximation" to Several choices of error to minimize: Squared error over nearest neighbors 1 X ( ( ) ? ^( )) ( ) 2

k f xq f x xq k f k E1 xq

x

k nearest nbrs of

f x

f x

E2 xq f x f x

2

K d xq ; x

:::

206

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

Global approximation to target function, in terms of linear combination of local approximations Used, e.g., for image classi cation A di erent kind of neural network Closely related to distance-weighted regression, but \eager" instead of \lazy"

207

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

f(x)

w0 1

w1

wk

... ...

where ( ) are the attributes describing instance , and ( )= + X ( ( )) One common choice for ( ( )) is

ai x x

k

a 1 (x) a2 (x)

a n (x)

f x

w0

u=1

wu K u d x u ; x

K u d xu ; x e

Ku d xu ; x

((

)) =

? 2 12

2 (x

u ;x)

208

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

Q1: What to use for each kernel function ( ( )) Scatter uniformly throughout instance space Or use training instances (re ects instance distribution) Q2: How to train weights (assume here Gaussian ) First choose variance (and perhaps mean) for each { e.g., use EM Then hold xed, and train linear output layer { e cient methods to t linear function

xu Ku d xu ; x Ku Ku Ku

209

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

Case-Based Reasoning

Can apply instance-based learning even when 6= < ! need di erent \distance" metric

X

n

Case-Based Reasoning is instance-based learning applied to instances with symbolic logic descriptions

((user-complaint error53-on-shutdown) (cpu-model PowerPC) (operating-system Windows) (network-connection PCIA) (memory 48meg) (installed-applications Excel Netscape VirusScan) (disk 1gig) (likely-cause ???))

210

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

CADET: 75 stored examples of mechanical devices each training example: h qualitative function, mechanical structurei new query: desired function, target value: mechanical structure for this function Distance metric: match qualitative function descriptions

211

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

A stored case: Tjunction pipe Structure: Q ,T 1 1 T = temperature Q = waterflow Q ,T 3 3 Q ,T 2 2 Function: Q 1 Q 2 T T 1 2 + + + T + 3 Q 3

+ C t + + C + f

Q c Q h T T c + +

+ +

Q m

h +

212

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

Instances represented by rich structural descriptions Multiple cases retrieved (and combined) to form solution to new problem Tight coupling between case retrieval and problem solving Bottom line: Simple matching of cases useful for tasks such as answering help-desk queries Area of ongoing research

213

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

Lazy: wait for query before generalizing -Nearest Neighbor, Case based reasoning

k

Eager: generalize before seeing query Radial basis function networks, ID3, Backpropagation, NaiveBayes, Does it matter? Eager learner must create global approximation Lazy learner can create many local approximations if they use same , lazy can represent more complex fns (e.g., consider = linear functions)

::: H H

214

lecture slides for textbook Machine Learning, c Tom M. Mitchell, McGraw Hill, 1997

- Intro Sockets cUploaded bykarma
- Multi Class SVM - File Exchange - MATLAB CentralUploaded byNamrata Singh
- Terrorist Network Analysis and Identification of Main Actors Using Machine Learning TechniquesUploaded byJawairia Rasheed
- syllabus_EECS940_S12Uploaded byPrateek Vasudev
- Class 01Uploaded byHabib Mrad
- 5244338Uploaded byCreativ Pinoy
- paper ID-28201441Uploaded byIJRAT
- Predicting New Collaborations in Academic Citation Networks of IEEE and ACM ConferencesUploaded bySaad
- Machine Learning Basic 2Uploaded byzgxfsbjbn
- UCI Machine Learning Repository_ Abalone Data SetUploaded byJoseph
- TermProj FRUploaded byDr-Mahmoud Al-Ewiwi
- IRJET-Classification of Liver Disease Based on US ImagesUploaded byIRJET Journal
- Smart CameraUploaded bygeorge ezar quiriado
- SCLUploaded bymusic2850
- Design and Development of Forest Fire Management SystemUploaded bysipij
- CON 02Uploaded byFalcão 69
- Splunk Lisa2014 Final VersionUploaded bysmjain
- lesson plan 4Uploaded byapi-265324068
- Start.docxUploaded bydaman.arneja143
- Vol21Issue1_2Uploaded byperl.freak
- Detection of Oriented and Skewed NumberUploaded byIJMAJournal
- The Data Science Machine, Or ‘How to Engineer Feature Engineering’Uploaded bysan
- Frontal Face Recognition From VideoUploaded bySahana Jp
- Deepfried at AUploaded byAnonymous Tj4OvGj5
- The long tail of the SingularityUploaded byWarren Smith QC (Quantum Cryptanalyst)
- Networking People FacebookUploaded bywhatisgoingon
- Artificial Intelligence _ Building Intelligent Systems, By Kulkarni, Parag, Joshi, Prachi __ 978-81-203-5046-5 __ Phi LearningUploaded byPHI Learning Pvt. Ltd.
- report templateUploaded byKesava Dass
- IRJET-Deep Learning Model to Predict Hardware PerformanceUploaded byIRJET Journal
- ReportUploaded byGunjan Khut

- ed-1-9-newUploaded byanandh_cdm
- Solar EnergyUploaded byanandh_cdm
- LinguisticsUploaded byanandh_cdm
- ed-2-5-newUploaded byanandh_cdm
- ed-2-1-new.pdfUploaded byanandh_cdm
- Laptop MaintenanceUploaded byanandh_cdm
- Verify Identities 2Uploaded byanandh_cdm
- ed-1-8-newUploaded byanandh_cdm
- ed-1-5-newUploaded byanandh_cdm
- BayesUploaded byanandh_cdm
- ed-1-12-newUploaded byanandh_cdm
- ed-1-4-newUploaded byanandh_cdm
- 100_windows7_shortcutsUploaded bymailtonani
- ed-2-4-newUploaded byanandh_cdm
- ed-1-6-7-newUploaded byanandh_cdm
- Revised Cbse Mains Key2012Uploaded byanandh_cdm
- ed-1-2-newUploaded byanandh_cdm
- Finding Structure in TimeUploaded bythangdaotao
- EC2 QP 510014Uploaded byanandh_cdm
- Xi Cpt Schedule FinalUploaded byanandh_cdm
- Famous AnglesUploaded byanandh_cdm
- IELTS Reading Answer SheetUploaded byskuppal
- IELTS Reading Answer SheetUploaded byskuppal
- Resume HandbookUploaded bynoelsaturno
- Electro Magnetic WavesUploaded bymicrodotcdm
- Mathematical Approaches to Comparative LinguisticsUploaded bytotalnarevolucija
- Letter WritingUploaded bymoomoojung
- ed-1-3-newUploaded byanandh_cdm
- WritingUploaded byanandh_cdm

- Md22s1Uploaded byAnonymous CKJIMFKS
- I-Vu Link Overview and Configuration 808-891r1Uploaded byabuMalak
- MODULE Excel MarksheetUploaded byshi3ting
- Jurnal Tinea Unguium - 1Uploaded byFazaKhilwanAmna
- Cell phone Based Voting MachineUploaded byHemant
- Soft Key RevealerUploaded byEroh Muthoat
- Advanced biocomposite from MAESSUploaded byFelipe Perissé Duarte Lopes
- Industrial Conveyors Belt Counting Object SystemUploaded byRavi Joshi
- k7 Question BookUploaded bysatourism
- Full Scale Rans ModelUploaded bySurya Chala Praveen
- Satellite OceanographyUploaded bythirumangai6
- 7519v1.0(G52-75191X1)(P45G45P43 Neo)Uploaded byphantomro
- SMP,MPP for OLAPUploaded byGirish Kalburgi
- CSFB-LTE-Feature-Training.pdfUploaded byEmerson Eduardo Rodrigues
- 30quasimpUploaded bybharath_sajja
- Spyglass CdcUploaded byRahul Sharma
- All the Little Pieces Distributed Systems With PHPUploaded bykaplumb_aga
- Vray TutorialUploaded byInelia Popescu
- BEGINNING MICROSOFT SMALL BASICUploaded bylilashcoco
- Vector Control of Induction Motor AbstractUploaded byBharath Chandra Reddy
- Drive Test Analysis - A Case StudyUploaded bygchivandire
- Apologetics 06Uploaded byTarlan Fisher
- PRoblem Test 2puUploaded byVinayak Bhat
- Lloyds Rules for Manufacture Testing and Certification of MaterialsUploaded bykatchani
- AWST-11-IL-02Uploaded byHerd Iman
- NEMESIS Executive SummaryUploaded bywavijunk
- TechRef_SoftstarterUploaded byАлишер Галиев
- [doi 10.1201%2F9781315316246-2] Perelmuter, Viktor -- Renewable Energy Systems (Simulation with Simulink® and SimPowerSystems™) __ Chapter 1 SimPowerSystems™ ModelsUploaded byTheodore Teddy J
- pre-honsUploaded byLimYC
- Piping Hydrostatic Testing (Overview)Uploaded byindusampath