Beruflich Dokumente
Kultur Dokumente
THE GENERAL-TO-SPECIFIC
ORDERING
WHAT IS A CONCEPT?
A Concept is a subset of subjects or events defined over
a larger set. For example, We refer to the set of
everything (all objects) as the set of things. Animals are a
subset of things,Anim
and birds are a subset of animals.
Thin
gs
Cars
als
Birds
WHAT IS A CONCEPT?
In more technical terms, a concept is a boolean-valued
function defined over this larger set.
For example, a function defined over all animals whose
value is true for birds and false for every other animal.
Sky
Airtemp
Humidit
y
Wind
Water
Forecast WaterSp
ort
Sunny
Warm
Normal
Strong
Warm
Same
Yes
Sunny
Warm
High
Strong
Warm
Same
Yes
Rainy
Cold
High
Strong
Warm
Change
No
Sunny
Warm
High
Strong
Cool
Change
Yes
Inductive Bias
Given the following examples:
Sunny,Warm,Normal,Strong,Cool,Change, yes
Cloudy,Warm,Normal,Light,Warm,Same, yes
Rainy,Warm,Normal,Light,Warm,Same, no
There is no hypothesis in our space that can represent
this concept. The closest we can come
is?,Warm,Normal,Light,Warm,Same
So, it is a biased learner
Unbiased Learner
We could define an unbiased hypothesis space by allowing arbitrary
conjunctions, disjunctions, and negations.
The number of possible hypothesis (ignoring semantic repetition)
is2|X|.
The Candidate-Elimination algorithm, if applied to this hypothesis
space, will be unable to generalize beyond the observed examples!
So, in order to learn the concept we would need to present every
single example.
If we use a partially-learned concept, thenexactly halfof the
hypotheses will vote positive and half negative for every unseen
example.
Formal Definition
Consider
1. concept learning algorithm L
2. instances X, target concept c
3. training examples Dc={x,c(x)}
4. let L(xi,Dc)denote the classification assigned to the instance x i by L
after training on data Dc.
Definition: The inductive bias of L is any minimal set of assertions B such
that for any target concept c and corresponding training examples of D
xiX(BDcxi)L(xi,Dc)
where A B means A logically entails B
For example, the inductive bias of the CandidateElimination algorithm (with voting) is the assumption
that the target concept is contained in the hypothesis
space.
That is, if we make that assumption then the
classificationfollows logically (by deduction).
CANDIDATE-ELIMINATION algorithm
diagram