Beruflich Dokumente
Kultur Dokumente
Going to infinity
Conclusion
Questions
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
g (v1 xt )
g (vn xt )
...
xt,1
...
Snowbird 2006
xt,d
Introduction
Going to infinity
Conclusion
Questions
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
...
h v1
O1
...
Op
...
h v2
...
h vk
xt,2
...
xt,d
xt,1
Nicolas Le Roux
Snowbird 2006
...
Introduction
Going to infinity
Conclusion
Questions
Going to infinity
b
f (x) =
X
j
wj g (vj x) + b b
f (x) =
Nicolas Le Roux
w (j) g (v (j) x) dj + b
j
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Going to infinity
b
f (x) =
X
j
wj g (vj x) + b b
f (x) =
w (j) g (v (j) x) dj + b
j
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Going to infinity
b
f (x) =
X
j
wj g (vj x) + b b
f (x) =
w (j) g (v (j) x) dj + b
j
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Going to infinity
b
f (x) =
X
j
wj g (vj x) + b b
f (x) =
w (j) g (v (j) x) dj + b
j
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Going to infinity
b
f (x) =
X
j
wj g (vj x) + b b
f (x) =
w (j) g (v (j) x) dj + b
j
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Going to infinity
b
f (x) =
X
j
wj g (vj x) + b b
f (x) =
w (j) g (v (j) x) dj + b
j
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Going to infinity
b
f (x) =
X
j
wj g (vj x) + b b
f (x) =
w (j) g (v (j) x) dj + b
j
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
wj
vj
b
f (x) =
g (v (u) x) du + b =
X
j
Nicolas Le Roux
Snowbird 2006
j
wj g (vj x) + b
Introduction
Going to infinity
Conclusion
Questions
wj
vj
b
f (x) =
g (v (u) x) du + b =
j
wj g (vj x) + b
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
V is a trajectory in Rd indexed by t
1
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
V is a trajectory in Rd indexed by t
1
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
V is a trajectory in Rd indexed by t
1
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
V is a trajectory in Rd indexed by t
1
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
wj
b
f (x) =
X
j
vj
wj
ln
(vj vj1 ) x
Nicolas Le Roux
cosh(vj x)
cosh(vj1 x)
Snowbird 2006
j
+b
Introduction
Going to infinity
Conclusion
Questions
wj
b
f (x) =
X
j
vj
wj
ln
(vj vj1 ) x
cosh(vj x)
cosh(vj1 x)
j
+b
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Rate of convergence
1
Z
f (x) 2a
f (x) b
Nicolas Le Roux
|(v (t) b
v (t)) x| dt
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Rate of convergence
1
Z
f (x) 2a
f (x) b
|(v (t) b
v (t)) x| dt
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Rate of convergence
1
Z
f (x) 2a
f (x) b
|(v (t) b
v (t)) x| dt
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Rate of convergence
1
Z
f (x) 2a
f (x) b
|(v (t) b
v (t)) x| dt
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
1.5
0.5
0.5
v (j) =
wv ,k g (vv ,k j + bv ,k )
Nicolas Le Roux
Snowbird 2006
10
Introduction
Going to infinity
Conclusion
Questions
v (j) =
wv ,k g (vv ,k j + bv ,k )
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
v (j) =
wv ,k g (vv ,k j + bv ,k )
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
v (j) =
wv ,k g (vv ,k j + bv ,k )
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
v (j) =
wv ,k g (vv ,k j + bv ,k )
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
v (j) =
wv ,k g (vv ,k j + bv ,k )
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
A bit of recursion
1
v (j) =
wv ,k g (vv ,k j + bv ,k )
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
A bit of recursion
1
v (j) =
wv ,k g (vv ,k j + bv ,k )
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
A bit of recursion
1
v (j) =
wv ,k g (vv ,k j + bv ,k )
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
A bit of recursion
1
v (j) =
wv ,k g (vv ,k j + bv ,k )
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
A bit of recursion
1
v (j) =
wv ,k g (vv ,k j + bv ,k )
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Summary
1
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Summary
1
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Summary
1
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Summary
1
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
2
3
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
2
3
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
2
3
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
2
3
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
2
3
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
wd = 103
2.270.13
58.270.50
7.710.10
1.720.11
1.670.10
1.720.10
wd = 106
1.800.08
58.540.27
7.780.21
2.090.09
2.780.25
3.040.26
Nicolas Le Roux
wd = 1012
1.800.08
58.540.27
7.780.21
2.100.09
3.330.35
4.390.49
Snowbird 2006
Test
4.07
58.29
12.31
4.07
3.58
3.77
Introduction
Going to infinity
Conclusion
Questions
wd = 103
5.51 0.22
77.55 0.40
10.51 0.46
3.64 0.10
3.01 0.12
3.15 0.09
Nicolas Le Roux
Snowbird 2006
Test
4.09
80.03
12.44
4.1
3.33
3.48
Introduction
Going to infinity
Conclusion
Questions
wd = 103
5.36 0.10
5.47 0.14
4.97 0.10
6.27 0.17
8.45 0.19
wd = 106
5.22 0.09
5.93 0.15
11.06 0.29
8.47 0.20
6.11 0.15
Nicolas Le Roux
Snowbird 2006
wd = 109
5.22 0.09
5.92 0.14
12.50 0.35
17.61 0.40
18.69 0.34
Test
5.5
5.8
5.3
6.63
9.25
Introduction
Going to infinity
Conclusion
Questions
Summary
Future work
Summary
1
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Summary
Future work
Summary
1
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Summary
Future work
Summary
1
In the extreme case where all the input weights are present,
we showed it is a kernel machine whose kernel can be
computed analytically and possesses no hyperparameter.
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Summary
Future work
Future work
1
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Summary
Future work
Future work
1
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Summary
Future work
Future work
1
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
Questions?
Nicolas Le Roux
Snowbird 2006
Computing
Introduction
Going to infinity
Conclusion
Questions
4
5
Nicolas Le Roux
Snowbird 2006
Introduction
Going to infinity
Conclusion
Questions
2
3
C (K , , b, ) = L(K + b, Y ) + 0 K
K
K
, c, b,
c + b, Y + c0 c
= L
c
c
c
c
c
= C (K , c, b, )
Nicolas Le Roux
Snowbird 2006