Sie sind auf Seite 1von 26

Cognitive Science 101

• Cognition is computation
• But what type?
• Fundamental question of research on the
human cognitive architecture

May 7, 2003 1 University of Amsterdam


Cognitive Architecture
• Rules operating on symbols
– grammar
– logic
• Spreading activation in simple processors
– massively interconnected in a large network
• Symbolic vs. Connectionist architecture?
• Integrated Connectionist/Symbolic (ICS)
Architecture
– Grammar: Phonology
• Architecture: defined by four components
May 7, 2003 2 University of Amsterdam
ƒ
dog+s G dgz

A The ICS Architecture


“dogs”

Representation Grammar G Function ƒ Algorithm A


σ
Constraints: optimal: 
σ
k NOCODA

æ t  k
æ t 
5

3
W (X  — B)

optimal: W

 Processing
2

0
0 1 2 3 4 5 6 7 8 9

De pt h- 1 Un it s

Activation Connection Harmony Spreading


(Learning)
Pattern Weights Optimization Activation
May 7, 2003 3 University of Amsterdam
Processing I: Activation
• Computational neuroscience

• Key sources
– Hopfield 1982, 1984
– Cohen and Grossberg 1983
– Hinton and Sejnowski 1983, 1986
– Smolensky 1983, 1986
– Geman and Geman 1984
– Golden 1986, 1988

May 7, 2003 4 University of Amsterdam


Processing I: Activation
Competitive Net Competitive Net

a10.2 –λ a2 da1
(–0.9)
0.3
  a1  i1  a2
0
-0.2
0.2
dt

0.5)
aa 22 (i(i 22 == 0.5)
0.1
-0.4
Harmony
-0.6 0
-0.8 0 0.2 0.4 0.6 0.8 1
1 -0.1
-1
0.5
-1.2 0 a1 -0.2
-0.5
-0.2

-0.5
0.1
0.4

-0.3
0.7
1

i1 a2 i2 a 1 (i 1 = 0.6)

(0.6) (0.5) Competitive Net

Processing — spreading 1

H (a )  a1—
activation  aoptimization:
i1 is 2 i2  a1 a2
0.8
0.6
a1

Harmony maximization
2 2 0.4
½( a1  a2 ) 0.2
H

H    a W a -0.2 1
-0.4
a2

Time

May 7, 2003 5 University of Amsterdam


ƒ
cat G
kæt
A The ICS Architecture

Representation Grammar G Function ƒ Algorithm A


σ
Constraints: optimal: 
σ
k NOCODA

æ t  k
æ t 
5

4
W (X  — B)

optimal H: W
3

0
0 1 2 3 4 5 6 7 8 9

De pt h- 1 Un it s

Activation Connection Harmony Spreading


Pattern Weights Optimization Activation

May 7, 2003 6 University of Amsterdam


Processing II: Optimization

• Cognitive psychology
• Key sources:
a1 –λ a2 – Hinton & Anderson 1981
(–0.9) – Rumelhart, McClelland, &
the PDP Group 1986

i1 i2 Competitive Net

(0.6) (0.5) 0.2


0
-0.2

Processing — spreading Harmony


-0.4
-0.6
-0.8

activation — is optimization: -1
-1.2 0
1
0.5
a1

Harmony maximization
-0.5
-0.2
-0.5

0.1
0.4
0.7

1
a2
May 7, 2003 7 University of Amsterdam
Processing II: Optimization
a1 and a2 must not
be simultaneously
active (strength: λ)
Harmony maximization is
a1 –λ a2 satisfaction of parallel, violable
(–0.9)
well-formedness constraints

a1 must be active CONFLIC a2 must be active


(strength: 0.6) T (strength: 0.5)
i1 i2 Competitive Net

(0.6) (0.5) 0.2


0
-0.2

Processing — spreading
Optimal
Harmony
-0.4
-0.6

activation0.79 –0.21
-0.8

compromise: — is optimization: -1
-1.2 0
1
0.5
a1

Harmony maximization
-0.5
-0.2
-0.5

0.1
0.4
0.7

1
a2
May 7, 2003 8 University of Amsterdam
Processing II: Optimization
• The search for an optimal state can employ
randomness
• Equations for units’ activation values have
random terms
– pr(a) ∝ eH(a)/T
– T (‘temperature’) ~ randomness  0 during search
– Boltzmann Machine (Hinton and Sejnowski 1983, 1986);
Harmony Theory (Smolensky 1983, 1986)

May 7, 2003 9 University of Amsterdam


ƒ
cat G
kæt
A The ICS Architecture

Representation Grammar G Function ƒ Algorithm A


σ
Constraints: optimal: 
σ
k NOCODA

æ t  k
æ t 
5

4
W (X  — B)

opt.constr. sat.: W
3

0
0 1 2 3 4 5 6 7 8 9

De pt h- 1 Un it s

Activation Connection Harmony Opt./ Spreading


Pattern Weights Constraint Activation
Satisfaction
May 7, 2003 10 University of Amsterdam
Two Fundamental Questions
 Harmony maximization is satisfaction of
parallel, violable constraints

2. What are the constraints?


Knowledge representation
Prior question:
1. What are the activation patterns — data
structures — mental representations —
evaluated by these constraints?

May 7, 2003 11 University of Amsterdam


Representation
 Symbolic theory
• Complex symbol structures
• Generative linguistics (Chomsky & Halle ’68 …)
 Particular linguistic representations
• Markedness Theory (Jakobson, Trubetzkoy, ’30s …)
 Good (well-formed) linguistic representations
 Connectionism (PDP)
• Distributed activation patterns
 ICS
• realization of (higher-level) complex symbolic
structures in distributed patterns of activation
over (lower-level) units
(‘tensor product representations’ etc.)

May 7, 2003
• will employ ‘local representations’
12
as well
University of Amsterdam
Representation
σ

k
æ t
{ f i / ri } i  i fi  ri
Activation patterns: cat and its constituents

σ/rε
k/r0
æ/r01
t/r11
[σ k [æ t]]
-1 4 9 14
Unit (Area = activation level)
May 7, 2003 13 University of Amsterdam
ƒ
cat G
kæt
A The ICS Architecture

Representation Grammar G Function ƒ Algorithm A


σ
Constraints: optimal: 
σ
k NOCODA

æ t  k
æ t 
5

4
W (X  — B)

opt.constr. sat.: W
3

0
0 1 2 3 4 5 6 7 8 9

De pt h- 1 Un it s

Activation Connection Harmony Opt./ Spreading


Pattern Weights Constraint Sat. Activation

May 7, 2003 14 University of Amsterdam


Constraints
σ NOCODA: A syllable has no coda [Maori]

k
* H(a[σ k [æ t]) =
æ t
‘cat’ violation –sNOCODA < 0
*
W

a[σ k [æ t ]] *

May 7, 2003 15 University of Amsterdam


ƒ
cat G
kæt
A The ICS Architecture

Representation Grammar G Function ƒ Algorithm A


σ
Constraints: optimal: 
σ
k NOCODA
æ t  k
æ t 
5

4
W (X  — B)

opt.constr. sat.: W
3

0
0 1 2 3 4 5 6 7 8 9

De pt h- 1 Un it s

Activation Connection Harmony Opt./ Spreading


Pattern Weights Constraint Sat. Activation

May 7, 2003 16 University of Amsterdam


ƒ
cat G
kæt
A The ICS Architecture
Constraint Interaction ??

Representation Grammar G Function ƒ Algorithm A


σ
Constraints: optimal: 
σ
k NOCODA
æ t
k
æ t 
5

4
W (X  — B)

opt.constr. sat.: W
3

0
0 1 2 3 4 5 6 7 8 9

De pt h- 1 Un it s

Activation Connection Harmony Opt./ Spreading


Pattern Weights Constraint Sat. Activation

May 7, 2003 17 University of Amsterdam


Constraint Interaction I
• Harmonic Grammar
– Legendre, Miyata, Smolensky 1990 et seq.

May 7, 2003 18 University of Amsterdam


Constraint Interaction I

σ The grammar generates the


representation that maximizes H:
H k
=H
this best-satisfies the constraints,
æ t given their differential strengths

Any formal language

=
H(k , σ) > 0 H(σ, t) < 0 can be so generated.

NOCODA
ONSET
 ij H ( c i , c j ) =   a Wa
May 7, 2003 19 University of Amsterdam
ƒ
cat G
kæt
A The ICS Architecture
Constraint Interaction I: HG

Representation Grammar G Function ƒ Algorithm A


σ
Constraints: optimal H: 
σ
k NOCODA
æ t
k
æ t 
5

4
W (X  — B)

opt.constr. sat.: W
3

0
0 1 2 3 4 5 6 7 8 9

De pt h- 1 Un it s

Activation Connection Harmony Opt./ Spreading


Pattern Weights Constraint Sat. Activation

May 7, 2003 20 University of Amsterdam


ƒ
cat G
kæt
A The ICS Architecture

Representation Grammar G Function ƒ Algorithm A


σ
Constraints: optimal:
k
æ t
NOCODA k
σ

æ t
?
5

4
W (X  — B)

opt.constr. sat.: W
3

0
0 1 2 3 4 5 6 7 8 9

De pt h- 1 Un it s

Activation Connection Harmony Opt./ Spreading


Pattern Weights Constraint Sat. Activation

May 7, 2003 21 University of Amsterdam


ƒ
cat G
kæt
A The ICS Architecture
HG Powerful: French syntax (Legendre, et al. 1990 et seq.)
Too powerful? Prince & Smolensky (1991 et seq.)
Representation Grammar G Function ƒ Algorithm A
σ
Constraints: optimal:
k
æ t
NOCODA k
σ

æ t

5

4
W (X  — B)

opt.constr. sat.: W
3

0
0 1 2 3 4 5 6 7 8 9

De pt h- 1 Un it s

Activation Connection Harmony Opt./ Spreading


Pattern Weights Constraint Sat. Activation

May 7, 2003 22 University of Amsterdam


ƒ
cat G
kæt
A The ICS Architecture
Constraint Interaction II

Representation Grammar G Function ƒ Algorithm A


σ
Constraints: optimal:
k
æ t
NOCODA k
σ


σ

æ
t
t

5

4
W (X  — B)

opt.constr. sat.: W
3

0
0 1 2 3 4 5 6 7 8 9

De pt h- 1 Un it s

Activation Connection Harmony Opt./ Spreading


Pattern Weights Constraint Sat. Activation

May 7, 2003 23 University of Amsterdam


ƒ
cat G
kæt
A The ICS Architecture
Constraint Interaction II: OT

Representation Grammar G Function ƒ Algorithm A


σ
Constraints: optimal:
Candidates STRESSHEAVY MAINSTRESSRIGHT


a. σHσ…σσ **…*
σ
k NOCODA b. σHσ…σσ
k
*

æ t æ t

4
W (X  — B)

opt.constr. sat.: W
3

0
0 1 2 3 4 5 6 7 8 9

De pt h- 1 Un it s

Activation Connection Harmony Opt./ Spreading


Pattern Weights Constraint Sat. Activation

May 7, 2003 24 University of Amsterdam


Constraint Interaction II: OT
• Strict domination
• “Grammars can’t count”
Candidates STRESSHEAVY MAINSTRESSRIGHT Harmony
a. σHσ…σσ **…* n(sMAINSTRESSRIGHT)
n n
b. σHσ…σσ * sSTRESSHEAVY
n

 Stress is on the initial heavy syllable iff


the number of light syllables n obeys
sSTRESSHEAVY
n  any number No way
sMAINSTRESSRIGHT

May 7, 2003 25 University of Amsterdam


Intro to OT

May 7, 2003 26 University of Amsterdam

Das könnte Ihnen auch gefallen