Beruflich Dokumente
Kultur Dokumente
Introduction
Construction
R functions
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
carolin.strobl@stat.uni-muenchen.de
useR! 2008, Dortmund
Introduction
Introduction
Construction
R functions
Random forests
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Introduction
Introduction
Construction
R functions
Random forests
Variable
importance
Summary
References
Introduction
Introduction
Construction
R functions
Random forests
Variable
importance
Summary
References
Introduction
Introduction
Construction
R functions
Random forests
Variable
importance
Summary
References
Introduction
Introduction
Construction
R functions
Random forests
Variable
importance
Summary
References
1
Start
p < 0.001
1
Start
p < 0.001
8
8
>8
12
2
n = 13
y = (0.308, 0.692)
>8
2
n = 15
y = (0.4, 0.6)
3
Age
p < 0.001
3
Start
p < 0.001
14
87
6
n = 16
y = (0.75, 0.25)
2
n = 38
y = (0.711, 0.289)
>5
9
n = 11
y = (0.364, 0.636)
2
Age
p < 0.001
> 12
81
3
n = 33
y = (1, 0)
3
Number
p < 0.001
> 81
4
Start
p < 0.001
12
5
n = 13
y = (0.385, 0.615)
>3
4
n = 25
y = (1, 0)
5
n = 18
y = (0.889, 0.111)
4
n = 11
y = (1, 0)
12
6
n = 12
y = (0.25, 0.75)
5
n = 31
y = (1, 0)
1
Start
p < 0.001
> 12
2
Age
p < 0.001
14
7
Number
p < 0.001
> 18
4
Number
p < 0.001
4
>3
8
9
n = 28
n = 21
y = (1, 0) y = (0.952, 0.048)
7
Start
p < 0.001
> 13
8
9
n = 11
n = 37
y = (0.818, 0.182) y = (1, 0)
>4
12
71
> 71
3
n = 15
y = (0.933, 0.067)
4
Start
p < 0.001
12
5
6
n = 12
n = 10
y = (0.417, 0.583)y = (0.2, 0.8)
> 12
81
8
5
Start
p < 0.001
> 81
13
7
n = 34
y = (1, 0)
1
Start
p < 0.001
12
5
Start
p < 0.001
12
3
4
6
n=9
n = 13
n = 12
y = (0.778, 0.222) y = (0.154, 0.846) y = (0.833, 0.167)
2
Age
p < 0.001
>3
136
6
n = 47
y = (1, 0)
> 136
7
n=8
y = (0.75, 0.25)
13
> 12
7
n = 47
y = (1, 0)
71
> 71
12
5
Start
p < 0.001
14
3
4
6
n = 15
n = 17
n = 17
y = (0.667, 0.333) y = (0.235, 0.765) y = (0.882, 0.118)
2
n = 28
y = (0.607, 0.393)
> 14
7
n = 32
y = (1, 0)
7
n = 10
y = (0.5, 0.5)
>3
3
Start
p < 0.001
6
n = 37
y = (0.865, 0.135)
> 13
4
n = 10
y = (0.8, 0.2)
5
n = 24
y = (1, 0)
1
Start
p < 0.001
1
Start
p < 0.001
> 12
Summary
>6
2
Number
p < 0.001
5
Age
p < 0.001
8
>8
Conditional
importance
1
Number
p < 0.001
>8
3
4
n = 12
n = 14
y = (0.667, 0.333) y = (0.143, 0.857)
> 12
5
6
n = 16
n = 15
y = (0.375, 0.625) y = (0.733, 0.267)
2
Start
p < 0.001
> 13
4
6
n = 16
n = 11
y = (0.188, 0.812) y = (0.818, 0.182)
1
Start
p < 0.001
>1
7
n = 35
y = (1, 0)
1
Start
p < 0.001
2
Age
p < 0.001
3
n = 20
y = (0.85, 0.15)
5
6
n = 14
n=9
y = (0.357, 0.643)
y = (0.111, 0.889)
> 14
2
Age
p < 0.001
>4
Variable
importance
6
n = 11
y = (0.818, 0.182)
13
4
Number
p < 0.001
7
n = 31
y = (0.806, 0.194)
> 125
1
Start
p < 0.001
1
Start
p < 0.001
> 27
>4
4
Age
p < 0.001
125
12
R functions
3
Number
p < 0.001
> 12
2
Age
p < 0.001
2
Start
p < 0.001
>1
5
n=9
y = (0.556, 0.444)
18
6
Start
p < 0.001
1
2
n=8
y = (0.375, 0.625)
>4
3
n = 10
y = (0.9, 0.1)
> 12
15 > 15
7
8
n = 12
n = 12
y = (0.833, 0.167) y = (1, 0)
1
Start
p < 0.001
7
n = 16
y = (1, 0)
7
n = 49
y = (1, 0)
> 68
3
Number
p < 0.001
> 13
1
Number
p < 0.001
12
27
68
5
Start
p < 0.001
13
5
n = 32
y = (1, 0)
1
Start
p < 0.001
3
n = 10
y = (1, 0)
> 87
4
n = 36
y = (1, 0)
> 14
4
n = 34
y = (0.882, 0.118)
> 12
2
Age
p < 0.001
Construction
1
Start
p < 0.001
2
n = 18
y = (0.5, 0.5)
> 12
3
Start
p < 0.001
14
4
n = 21
y = (0.905, 0.095)
>8
3
Start
p < 0.001
12
> 14
5
n = 32
y = (1, 0)
> 12
4
n = 18
y = (0.833, 0.167)
5
Number
p < 0.001
3
6
n = 30
y = (1, 0)
>3
7
n = 15
y = (0.933, 0.067)
References
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
I
I
Variable
importance
ntree trees
Conditional
importance
Summary
References
I
I
Variable
importance
ntree trees
Conditional
importance
Summary
References
Random forests in R
Introduction
Construction
R functions
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
1
Start
p < 0.001
1
Start
p < 0.001
8
8
>8
12
2
n = 13
y = (0.308, 0.692)
>8
2
n = 15
y = (0.4, 0.6)
3
Age
p < 0.001
3
Start
p < 0.001
14
87
6
n = 16
y = (0.75, 0.25)
2
n = 38
y = (0.711, 0.289)
>5
9
n = 11
y = (0.364, 0.636)
2
Age
p < 0.001
> 12
81
3
n = 33
y = (1, 0)
3
Number
p < 0.001
> 81
4
Start
p < 0.001
12
5
n = 13
y = (0.385, 0.615)
>3
4
n = 25
y = (1, 0)
5
n = 18
y = (0.889, 0.111)
4
n = 11
y = (1, 0)
12
6
n = 12
y = (0.25, 0.75)
5
n = 31
y = (1, 0)
1
Start
p < 0.001
> 12
2
Age
p < 0.001
14
7
Number
p < 0.001
> 18
4
Number
p < 0.001
4
>3
8
9
n = 28
n = 21
y = (1, 0) y = (0.952, 0.048)
7
Start
p < 0.001
> 13
8
9
n = 11
n = 37
y = (0.818, 0.182) y = (1, 0)
>4
12
71
> 71
3
n = 15
y = (0.933, 0.067)
4
Start
p < 0.001
12
5
6
n = 12
n = 10
y = (0.417, 0.583)y = (0.2, 0.8)
> 12
81
8
5
Start
p < 0.001
> 81
13
7
n = 34
y = (1, 0)
1
Start
p < 0.001
12
5
Start
p < 0.001
12
3
4
6
n=9
n = 13
n = 12
y = (0.778, 0.222) y = (0.154, 0.846) y = (0.833, 0.167)
2
Age
p < 0.001
>3
136
6
n = 47
y = (1, 0)
> 136
7
n=8
y = (0.75, 0.25)
13
> 12
7
n = 47
y = (1, 0)
71
> 71
12
5
Start
p < 0.001
14
3
4
6
n = 15
n = 17
n = 17
y = (0.667, 0.333) y = (0.235, 0.765) y = (0.882, 0.118)
2
n = 28
y = (0.607, 0.393)
> 14
7
n = 32
y = (1, 0)
7
n = 10
y = (0.5, 0.5)
>3
3
Start
p < 0.001
6
n = 37
y = (0.865, 0.135)
> 13
4
n = 10
y = (0.8, 0.2)
5
n = 24
y = (1, 0)
1
Start
p < 0.001
1
Start
p < 0.001
> 12
Summary
>6
2
Number
p < 0.001
5
Age
p < 0.001
8
>8
Conditional
importance
1
Number
p < 0.001
>8
3
4
n = 12
n = 14
y = (0.667, 0.333) y = (0.143, 0.857)
> 12
5
6
n = 16
n = 15
y = (0.375, 0.625) y = (0.733, 0.267)
2
Start
p < 0.001
> 13
4
6
n = 16
n = 11
y = (0.188, 0.812) y = (0.818, 0.182)
1
Start
p < 0.001
>1
7
n = 35
y = (1, 0)
1
Start
p < 0.001
2
Age
p < 0.001
3
n = 20
y = (0.85, 0.15)
5
6
n = 14
n=9
y = (0.357, 0.643)
y = (0.111, 0.889)
> 14
2
Age
p < 0.001
>4
Variable
importance
6
n = 11
y = (0.818, 0.182)
13
4
Number
p < 0.001
7
n = 31
y = (0.806, 0.194)
> 125
1
Start
p < 0.001
1
Start
p < 0.001
> 27
>4
4
Age
p < 0.001
125
12
R functions
3
Number
p < 0.001
> 12
2
Age
p < 0.001
2
Start
p < 0.001
>1
5
n=9
y = (0.556, 0.444)
18
6
Start
p < 0.001
1
2
n=8
y = (0.375, 0.625)
>4
3
n = 10
y = (0.9, 0.1)
> 12
15 > 15
7
8
n = 12
n = 12
y = (0.833, 0.167) y = (1, 0)
1
Start
p < 0.001
7
n = 16
y = (1, 0)
7
n = 49
y = (1, 0)
> 68
3
Number
p < 0.001
> 13
1
Number
p < 0.001
12
27
68
5
Start
p < 0.001
13
5
n = 32
y = (1, 0)
1
Start
p < 0.001
3
n = 10
y = (1, 0)
> 87
4
n = 36
y = (1, 0)
> 14
4
n = 34
y = (0.882, 0.118)
> 12
2
Age
p < 0.001
Construction
1
Start
p < 0.001
2
n = 18
y = (0.5, 0.5)
> 12
3
Start
p < 0.001
14
4
n = 21
y = (0.905, 0.095)
>8
3
Start
p < 0.001
12
> 14
5
n = 32
y = (1, 0)
> 12
4
n = 18
y = (0.833, 0.167)
5
Number
p < 0.001
3
6
n = 30
y = (1, 0)
>3
7
n = 15
y = (0.933, 0.067)
References
Variable
Gini importance
importance
column: MeanDecreaseGini
importance(obj, type=2)
Summary
References
permutation importance
Construction
R functions
column: MeanDecreaseAccuracy
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
importance(obj, type=1)
I
References
Introduction
Construction
R functions
Variable
P
VI (t) (xj ) =
(t)
I yi = yi
(t)
B
(t)
iB
(t)
iB
(t)
I yi = yi,j
(t)
B
importance
Tests for variable
importance
Conditional
importance
Summary
(t)
yi
(t)
References
Variable
importance
1. raw importance
Pntree
VI (xj ) =
VI (t) (xj )
ntree
Summary
t=1
References
Variable
importance
VI (xj )
ntree
Summary
= zj
References
Introduction
Construction
R functions
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Introduction
Construction
R functions
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Introduction
Construction
R functions
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Introduction
Construction
R functions
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Variable
importance
Tests for variable
importance
as.
zj N(0, 1)
Conditional
importance
Summary
References
Raw importance
Introduction
Construction
R functions
Variable
sample size
importance
100
200
500
mean importance
ntree = 200
mean importance
ntree = 100
mean importance
ntree = 500
Conditional
importance
Summary
References
0.0
0.1
0.2
0.3
0.4
0.0
0.1
0.2
relevance
0.3
0.4
0.0
0.1
0.2
0.3
0.4
sample size
Construction
100
200
500
zscore
ntree = 200
zscore
ntree = 100
R functions
zscore
ntree = 500
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
0.0
0.1
0.2
0.3
0.4
0.0
0.1
power
ntree = 100
0.0
0.1
0.2
0.2
0.3
0.4
0.0
0.1
power
ntree = 200
0.3
0.4
0.0
0.1
0.2
relevance
0.2
0.3
0.4
References
power
ntree = 500
0.3
0.4
0.0
0.1
0.2
0.3
0.4
Findings
Introduction
Construction
R functions
Variable
importance
increase in ntree
Summary
References
Introduction
Construction
R functions
Variable
obs
Xj
1
..
.
y1
..
.
xj (1),j
..
.
z1
..
.
i
..
.
yi
..
.
xj (i),j
..
.
zi
..
.
yn
xj (n),j
zn
H0 : Xj Y , Z or Xj Y Xj Z
H
importance
Tests for variable
importance
Conditional
importance
Summary
References
Introduction
Construction
R functions
Variable
importance
Summary
References
Introduction
Construction
R functions
Variable
importance
Summary
References
obs
Xj
y1
xj|Z =a (1),j
z1 = a
y3
xj|Z =a (3),j
z3 = a
27
y27
xj|Z =a (27),j
z27 = a
y6
xj|Z =b (6),j
z6 = b
14
y14
xj|Z =b (14),j
z14 = b
33
..
.
y33
..
.
xj|Z =b (33),j
..
.
z33 = b
..
.
H0 : Xj Y |Z
P(Y , Xj |Z )
H0
P(Y |Z ) P(Xj |Z )
or P(Y |Xj , Z )
H0
P(Y |Z )
Construction
R functions
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Technically
Introduction
Construction
R functions
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Technically
Introduction
Construction
R functions
I
I
Variable
importance
Conditional
importance
Summary
References
Technically
Introduction
Construction
R functions
I
I
Variable
importance
Conditional
importance
Summary
References
Technically
Introduction
Construction
R functions
I
I
Variable
importance
Conditional
importance
Summary
References
Simulation study
I
I
Introduction
i.i.d.
Construction
R functions
Variable
0.9
0.9
0.9
0 0
0.9
0.9
0.9
0.9
0.9
0.9
0.9
0.9
0.9
0
..
.
0
..
.
0
..
.
0
..
.
0 0
0 0
0 0
1 0
.. . .
. 0
.
importance
Summary
References
Xj
X1
X2
X3
X4
X5
X6
X7
X8
X12
-5
-5
-2
Results
Construction
R functions
15
mtry = 1
25
Introduction
Variable
0 5
importance
30
Summary
References
10
11
12
mtry = 8
0 10
mtry = 3
50
Conditional
importance
20 40 60 80
variable
variable
Peptide-binding data
Introduction
Construction
Variable
0.005
importance
Tests for variable
importance
Conditional
importance
Summary
0.005
References
conditional
conditional
unconditional
R functions
h2y8
flex8
*
pol3
Summary
Introduction
Construction
R functions
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Summary
if your predictor variables are of different types:
use cforest (pkg: party) with default option controls =
Introduction
Construction
R functions
cforest unbiased()
with permutation importance varimp(obj)
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Summary
if your predictor variables are of different types:
use cforest (pkg: party) with default option controls =
Introduction
Construction
R functions
cforest unbiased()
with permutation importance varimp(obj)
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Summary
if your predictor variables are of different types:
use cforest (pkg: party) with default option controls =
Introduction
Construction
R functions
cforest unbiased()
with permutation importance varimp(obj)
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Introduction
Construction
R functions
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Introduction
Construction
R functions
Variable
importance
Tests for variable
importance
Conditional
importance
Summary
References
Introduction
Construction
R functions
Variable
importance
Conditional
importance
Summary
References