INTRODUCTION CSE463 2014 Fall

Machine Learning
Course number: CSE48301
Instructor: Jaesik Choi
http://pail.unist.ac.kr/classes
1
Slides are courtesy of
Dan Roth, University of Illinois, Urbana-Champaign
http://l2r.cs.uiuc.edu/~danr/Teaching/CS446-12/
INTRODUCTION CSE463 2014 Fall
Machine Learning
Mon, Weds: 1pm-2:15pm at EB2 T205
Office hours: Thrsday 4pm-5pm [EB2 501-10]
TA: Yunseong Hwang (kika1492@unist.ac.kr)
Grading:
H/W - Reading Turing Award Lectures (10%)
H/W - 2 Programming Assignments (Kaggle) (10%)
- Decision Tree and Deep Belief Network
Midterm Exam (25%)
Final Exam (25%)
Project Kaggle competition or your own (30%)
Textbook: Tom Mitchell Machine Learning
Lecture notes /Literature
2
INTRODUCTION CSE463 2014 Fall
Machine Learning: Today
What is Learning?
Who are you?
What is this course about?
3
INTRODUCTION CSE463 2014 Fall
Who are you?
What is Learning
Who are you?
4
INTRODUCTION CSE463 2014 Fall
Comprehension
(ENGLAND, June, 1989) - Christopher Robin is alive and well. He lives in
England. He is the same person that you read about in the book, Winnie the
Pooh. As a boy, Chris lived in a pretty home called Cotchfield Farm. When
Chris was three years old, his father wrote a poem about him. The poem was
printed in a magazine for others to read. Mr. Robin then wrote a book. He
made up a fairy tale land where Chris lived. His friends were animals. There was
a bear called Winnie the Pooh. There was also an owl and a young pig, called a
piglet. All the animals were stuffed toys that Chris owned. Mr. Robin made
them come to life with his words. The places in the story were all near
Cotchfield Farm. Winnie the Pooh was written in 1925. Children still love to
read about Christopher Robin and his animal friends. Most people don't know
he is a real person who is grown now. He has written two books of his own.
They tell what it is like to be famous.
1. Christopher Robin was born in England. 2. Winnie the Pooh is a title of a book.
3. Christopher Robins dad was a magician. 4. Christopher Robin must be at least 65 now.
This is an Inference Problem; where is the learning?
5
INTRODUCTION CSE463 2014 Fall
Ambiguity Resolution
Can I have a peace of cake ? piece
...Nissan Car and truck plant is
divide life into plant and animal kingdom
Buy a car with a steering wheel (his money)
(This Art) (can N) (will MD) (rust V) V,N,N
The dog bit the kid. He was taken to a veterinarian
hospital
Learn a function that maps observations in the domain
to one of several categories or <.
6
INTRODUCTION CSE463 2014 Fall
Applications
Knowledge Representation
Natural Language Processing
Autonomous Car
Controlling Robots
Computer Vision
7
INTRODUCTION CSE463 2014 Fall
How does ML change the world?
8
Common sense/knowledge representation
E.g., IBM Watson for Q/A problems.
http://www.youtube.com/watch?v=WFR3lOm_xhE
INTRODUCTION CSE463 2014 Fall
How does ML change the world?
9
Natural Language Processing
Personal assistants: e.g., Apple Siri / Google Now
http://www.youtube.com/watch?v=pPqliPzHYyc
http://www.youtube.com/watch?v=UEVpy8NvAlA
INTRODUCTION CSE463 2014 Fall
How does ML change the world?
10
Controlling Autonomous car
E.g., Google Autonomous cars.
http://www.youtube.com/watch?v=cdgQpa1pUUE
INTRODUCTION CSE463 2014 Fall
How does ML change the world?
11
Learning actions for robots
E.g., Baxter Robots
https://www.youtube.com/watch?v=ghTlux5Dv90
INTRODUCTION CSE463 2014 Fall
How does ML change the world?
12
Computer Vision
E.g., Face/Object Detection with Google Glass
http://www.youtube.com/watch?v=Z-siJX3LNO0
INTRODUCTION CSE463 2014 Fall
The Role of Learning
Learning is at the core of
Understanding High Level Cognition
Performing knowledge intensive inferences
Building adaptive, intelligent systems
Dealing with messy, real world data
Learning has multiple purposes
Knowledge Acquisition
Integration of various knowledge sources to ensure robust behavior
Adaptation (human, systems)
13
INTRODUCTION CSE463 2014 Fall
Learning = Generalization
Herbert Simon -
Learning denotes changes in the system that are
adaptive in the sense that they enable the system to
do the task or tasks drawn from the same population
more efficiently and more effectively the next time.
The ability to perform a task in a situation which
has never been encountered before
14
INTRODUCTION CSE463 2014 Fall
Learning = Generalization
Classification
Medical diagnosis; credit card applications; hand-written
letters
Planning and acting
Navigation; game playing (chess, backgammon); driving a
car
Skills
Balancing a pole; playing tennis
Common sense reasoning
Natural language interactions
Generalization depends on Representation as much
as it depends on the Algorithm used.
The ability to perform a task in a situation
which has never been encountered before
15
INTRODUCTION CSE463 2014 Fall
Why Study Learning?
Computer systems with new capabilities.
Develop systems that are too difficult to impossible to
construct manually .
Develop systems that can automatically adapt and
customize themselves to the needs of the individual
users through experience.
Discover knowledge and patterns in databases,
database mining, e.g. discovering purchasing patterns
for marketing purposes.
16
INTRODUCTION CSE463 2014 Fall
Why Study Learning?
Computer systems with new capabilities.
Understand human and biological learning
Understanding teaching better.
17
INTRODUCTION CSE463 2014 Fall
Why Study Learning?
Computer systems with new capabilities.
Understand human and biological learning
Understanding teaching better.
Time is right.
Initial algorithms and theory in place.
Growing amounts of on-line data
Computational power available.
Necessity: many things we want to do cannot be done
by programming.
18
INTRODUCTION CSE463 2014 Fall
Learning is the future
Learning techniques will be a basis for every application that
involves a connection to the messy real world
Basic learning algorithms are ready for use in applications
today
Prospects for broader future applications make for exciting
fundamental research and development opportunities
Many unresolved issues Theory and Systems
19
INTRODUCTION CSE463 2014 Fall
Work in Machine Learning
Artificial Intelligence; Theory; Experimental CS
Makes Use of:
Probability and Statistics; Linear Algebra; Statistics; Theory of Computation;
Related to:
Philosophy, Psychology (cognitive, developmental), Neurobiology,
Linguistics
Has applications in:
AI (natural Language; Vision; Planning; HCI)
Engineering (Agriculture; Civil; )
Computer Science (Compilers; Architecture; Systems; data bases)
20
INTRODUCTION CSE463 2014 Fall
Work in Machine Learning
Artificial Intelligence; Theory; Experimental CS
Makes Use of:
Probability and Statistics; Linear Algebra; Statistics; Theory of Computation;
Related to:
Philosophy, Psychology (cognitive, developmental), Neurobiology,
Linguistics
Has applications in:
AI (natural Language; Vision; Planning; HCI)
Engineering (Agriculture; Civil; )
Computer Science (Compilers; Architecture; Systems; data bases)
Very active field; All of AI today is doing Machine Learning
What to teach?
The fundamental paradigms
Some of the most important algorithmic ideas
Modeling
And: What we
dont know
What YOU care
about
21
INTRODUCTION CSE463 2014 Fall
Course Overview
Introduction: Basic problems and questions
A detailed examples: Linear threshold units
Hands on classification
Two Basic Paradigms:
Probably Approximately Correct (Risk Minimization)
Bayesian theory
Learning Protocols:
Supervised; Unsupervised; Semi-supervised
Algorithms
Decision Trees and Random Forests
Linear Threshold Units (Winnow; Perceptron; Boosting; SVMs; Kernels)
Logistic Regression
Neural Networks (Backpropagation) and Deep Networks
Probabilistic Representations (nave Bayes; Bayesian trees; Densities)
Unsupervised /Semi supervised: EM
Clustering (K-means); Dimensionality Reduction
Who knows DTs ?
Who knows NNs ?
22
INTRODUCTION CSE463 2014 Fall
Machine Learning
What do you need to know:
Probability Theory
Linear Algebra
Programming (Java; your favorite language; some Matlab)
Who is the class for?
Future Machine Learning researchers/data scientists
23
INTRODUCTION CSE463 2014 Fall
Course: Policies
Cheating
No.
Homework:
Collaboration is encouraged;
But, you have to write your own solution/program.
Late Policy:
You have a credit of 4 days (4*24hours); Thats it.
Grading:
Possibly separate for grads/undergrads.
25%-midterm; 25%-final; HW: 20%; Projects: 30%
Questions?
24
INTRODUCTION CSE463 2014 Fall
Plagiarism Why is it bad?
My educational principle
It will prevent you becoming a good person and leader.
Such compromises will demote your creativity for great
achievement.
A quote from Madison Sarratt, dean of Vanderbilt
University
Today I am going to give you two examinations, one in
trigonometry and one in honesty. I hope you will pass them
both, but if you must fail one, let it be trigonometry, for
there are many good [people] in this world today who
cannot pass an examination in trigonometry, but there are
no good [people] in the world who cannot pass an
examination in honesty.
25
INTRODUCTION CSE463 2014 Fall
H/W Reading Turing Award Lectures
What is Turing Award
Turing Award is a prize given by the ACM (Association for
Computing Machinery). The Turing Award is recognized as
Nobel Prize of computing.
Why?
Learning mind sets from great researchers.
What?
Write a brief biography of a Turing Award winner and
summarize seminal paper (or Turing Award lecture).
How?
We will give you some questions.
26
INTRODUCTION CSE463 2014 Fall
H/W Reading Turing Award Lectures
Marvin Minsky (1969)
Analysis of Perceptron and Artificial Neural Networks
Leslie Gabriel Valiant (2010)
Probably Approximately Correct (PAC)
Judea Pearl (2011)
Bayesian Networks (Causal reasoning)
27
INTRODUCTION CSE463 2014 Fall
H/W Reading Turing Award Lectures
Marvin Minsky (1969)
Analysis of Perceptron and Artificial Neural Networks
Leslie Gabriel Valiant (2010)
Probably Approximately Correct (PAC)
Judea Pearl (2011)
Bayesian Networks (Causal reasoning)
28
Q: How about the other winners?
McCarthy, John (1971), Newell, Allen (1975),
Simon, Herbert ("Herb") Alexander (1975),
Feigenbaum, Edward A ("Ed") (1994), Reddy,
Dabbala Rajagopal ("Raj") (1994)
A: We will see in an Artificial Intelligence course.
INTRODUCTION CSE463 2014 Fall
Project Kaggle.com
29
INTRODUCTION CSE463 2014 Fall
Titanic: Machine Learning from Disaster
30
Description: The sinking of the RMS Titanic is one of the most infamous
shipwrecks in history. On April 15, 1912, during her maiden voyage, the
Titanic sank after colliding with an iceberg, killing 1502 out of 2224
passengers and crew. One of the reasons that were not enough
lifeboats for the passengers and crew. Although there was some
element of luck , some groups of people were more likely to survive
than others, such as women, children, and the upper-class.
Problem: In this contest, we ask you to complete the analysis of what
sorts of people were likely to survive. In particular, we ask you to apply
the tools of machine learning to predict which passengers survived the
tragedy.
INTRODUCTION CSE463 2014 Fall
Titanic: Machine Learning from Disaster
31
Data:
INTRODUCTION CSE463 2014 Fall
Dogs vs. Cats
32
Description: Create an algorithm to distinguish dogs from cats. In this
competition, you'll write an algorithm to classify whether images contain
either a dog or a cat. This is easy for humans, dogs, and cats. Your
computer will find it a bit more difficult.
Problem:
INTRODUCTION CSE463 2014 Fall
Dogs vs. Cats
33
Data:
Output: