Beruflich Dokumente
Kultur Dokumente
• Cognition is computation
• But what type?
• Fundamental question of research on the
human cognitive architecture
3
W (X — B)
optimal: W
Processing
2
0
0 1 2 3 4 5 6 7 8 9
De pt h- 1 Un it s
• Key sources
– Hopfield 1982, 1984
– Cohen and Grossberg 1983
– Hinton and Sejnowski 1983, 1986
– Smolensky 1983, 1986
– Geman and Geman 1984
– Golden 1986, 1988
a10.2 –λ a2 da1
(–0.9)
0.3
a1 i1 a2
0
-0.2
0.2
dt
0.5)
aa 22 (i(i 22 == 0.5)
0.1
-0.4
Harmony
-0.6 0
-0.8 0 0.2 0.4 0.6 0.8 1
1 -0.1
-1
0.5
-1.2 0 a1 -0.2
-0.5
-0.2
-0.5
0.1
0.4
-0.3
0.7
1
i1 a2 i2 a 1 (i 1 = 0.6)
Processing — spreading 1
H (a ) a1—
activation aoptimization:
i1 is 2 i2 a1 a2
0.8
0.6
a1
Harmony maximization
2 2 0.4
½( a1 a2 ) 0.2
H
H a W a -0.2 1
-0.4
a2
Time
4
W (X — B)
optimal H: W
3
0
0 1 2 3 4 5 6 7 8 9
De pt h- 1 Un it s
• Cognitive psychology
• Key sources:
a1 –λ a2 – Hinton & Anderson 1981
(–0.9) – Rumelhart, McClelland, &
the PDP Group 1986
i1 i2 Competitive Net
activation — is optimization: -1
-1.2 0
1
0.5
a1
Harmony maximization
-0.5
-0.2
-0.5
0.1
0.4
0.7
1
a2
May 7, 2003 7 University of Amsterdam
Processing II: Optimization
a1 and a2 must not
be simultaneously
active (strength: λ)
Harmony maximization is
a1 –λ a2 satisfaction of parallel, violable
(–0.9)
well-formedness constraints
Processing — spreading
Optimal
Harmony
-0.4
-0.6
activation0.79 –0.21
-0.8
compromise: — is optimization: -1
-1.2 0
1
0.5
a1
Harmony maximization
-0.5
-0.2
-0.5
0.1
0.4
0.7
1
a2
May 7, 2003 8 University of Amsterdam
Processing II: Optimization
• The search for an optimal state can employ
randomness
• Equations for units’ activation values have
random terms
– pr(a) ∝ eH(a)/T
– T (‘temperature’) ~ randomness 0 during search
– Boltzmann Machine (Hinton and Sejnowski 1983, 1986);
Harmony Theory (Smolensky 1983, 1986)
4
W (X — B)
opt.constr. sat.: W
3
0
0 1 2 3 4 5 6 7 8 9
De pt h- 1 Un it s
May 7, 2003
• will employ ‘local representations’
12
as well
University of Amsterdam
Representation
σ
k
æ t
{ f i / ri } i i fi ri
Activation patterns: cat and its constituents
σ/rε
k/r0
æ/r01
t/r11
[σ k [æ t]]
-1 4 9 14
Unit (Area = activation level)
May 7, 2003 13 University of Amsterdam
ƒ
cat G
kæt
A The ICS Architecture
4
W (X — B)
opt.constr. sat.: W
3
0
0 1 2 3 4 5 6 7 8 9
De pt h- 1 Un it s
k
* H(a[σ k [æ t]) =
æ t
‘cat’ violation –sNOCODA < 0
*
W
a[σ k [æ t ]] *
4
W (X — B)
opt.constr. sat.: W
3
0
0 1 2 3 4 5 6 7 8 9
De pt h- 1 Un it s
4
W (X — B)
opt.constr. sat.: W
3
0
0 1 2 3 4 5 6 7 8 9
De pt h- 1 Un it s
=
H(k , σ) > 0 H(σ, t) < 0 can be so generated.
NOCODA
ONSET
ij H ( c i , c j ) = a Wa
May 7, 2003 19 University of Amsterdam
ƒ
cat G
kæt
A The ICS Architecture
Constraint Interaction I: HG
4
W (X — B)
opt.constr. sat.: W
3
0
0 1 2 3 4 5 6 7 8 9
De pt h- 1 Un it s
æ t
?
5
4
W (X — B)
opt.constr. sat.: W
3
0
0 1 2 3 4 5 6 7 8 9
De pt h- 1 Un it s
æ t
5
4
W (X — B)
opt.constr. sat.: W
3
0
0 1 2 3 4 5 6 7 8 9
De pt h- 1 Un it s
kæ
σ
æ
t
t
5
4
W (X — B)
opt.constr. sat.: W
3
0
0 1 2 3 4 5 6 7 8 9
De pt h- 1 Un it s
a. σHσ…σσ **…*
σ
k NOCODA b. σHσ…σσ
k
*
æ t æ t
4
W (X — B)
opt.constr. sat.: W
3
0
0 1 2 3 4 5 6 7 8 9
De pt h- 1 Un it s