Beruflich Dokumente
Kultur Dokumente
Pawan Goyal
CSE, IIT Kharagpur
Distributional Semantics
1 / 24
Introduction
1, 3, 4, . . .
Distributional Semantics
2 / 24
Introduction
1, 3, 4, . . .
I, III, IV, . . .
Distributional Semantics
2 / 24
Introduction
1, 3, 4, . . .
I, III, IV, . . .
What is Semantics?
Distributional Semantics
2 / 24
Introduction
1, 3, 4, . . .
I, III, IV, . . .
What is Semantics?
The study of meaning: Relation between symbols and their denotata.
Distributional Semantics
2 / 24
Introduction
1, 3, 4, . . .
I, III, IV, . . .
What is Semantics?
The study of meaning: Relation between symbols and their denotata.
John told Mary that the train moved out of the station at 3 oclock.
Distributional Semantics
2 / 24
Finding the underlying functional relations among various entities and events
Distributional Semantics
3 / 24
Computational Semantics
Distributional Semantics
4 / 24
Computational Semantics
Computational Semantics
The study of how to automate the process of constructing and reasoning with
meaning representations of natural language expressions.
Distributional Semantics
4 / 24
Computational Semantics
Computational Semantics
The study of how to automate the process of constructing and reasoning with
meaning representations of natural language expressions.
Distributional Semantics
4 / 24
Computational Semantics
Computational Semantics
The study of how to automate the process of constructing and reasoning with
meaning representations of natural language expressions.
Distributional Semantics
4 / 24
Computational Semantics
Computational Semantics
The study of how to automate the process of constructing and reasoning with
meaning representations of natural language expressions.
Distributional Semantics
4 / 24
Distributional Hypothesis
Distributional Semantics
5 / 24
Distributional Hypothesis
Distributional Semantics
5 / 24
Distributional Hypothesis
Distributional Semantics
5 / 24
Distributional Hypothesis
Distributional Semantics
5 / 24
Distributional Hypothesis
Distributional Semantics
5 / 24
Distributional Semantics
6 / 24
Distributional Semantics
6 / 24
Distributional Semantics
6 / 24
Contextual representation
A words contextual representation is an abstract cognitive structure that
accumulates from encounters with the word in various linguistic contexts.
Distributional Semantics
7 / 24
Contextual representation
A words contextual representation is an abstract cognitive structure that
accumulates from encounters with the word in various linguistic contexts.
Distributional Semantics
7 / 24
Contextual representation
A words contextual representation is an abstract cognitive structure that
accumulates from encounters with the word in various linguistic contexts.
Distributional Semantics
7 / 24
Contextual representation
A words contextual representation is an abstract cognitive structure that
accumulates from encounters with the word in various linguistic contexts.
Distributional Semantics
7 / 24
Distributional Semantics
8 / 24
Distributional Semantics
8 / 24
Alternative names
I
I
I
I
I
corpus-based semantics
statistical semantics
geometrical models of meaning
vector semantics
word space models
Distributional Semantics
8 / 24
Distributional Semantics
9 / 24
Vector Space
Distributional Semantics
10 / 24
Vector Space
Distributional Semantics
10 / 24
Vector Space
Distributional Semantics
10 / 24
Word Space
Small Dataset
An automobile is a wheeled motor vehicle used for transporting passengers .
A car is a form of transport , usually with four wheels and the capacity to carry around
five passengers .
Transport for the London games is limited , with spectators strongly advised to avoid
the use of cars .
The London 2012 soccer tournament began yesterday , with plenty of goals in the
opening matches .
Giggs scored the first goal of the football tournament at Wembley , North London .
Bellamy was largely a passenger in the football match , playing no part in either goal .
Distributional Semantics
11 / 24
Distributional Semantics
12 / 24
Distributional Semantics
12 / 24
Distributional Semantics
12 / 24
Count number of times the target word co-occurs with the context words:
co-occurrence matrix
Distributional Semantics
12 / 24
Count number of times the target word co-occurs with the context words:
co-occurrence matrix
Build vectors out of (a function of) these co-occurrence counts
Distributional Semantics
12 / 24
automobile
car
soccer
football
wheel
1
1
0
0
transport
1
2
0
0
passenger
1
1
0
1
tournament
0
0
1
1
Distributional Semantics
London
0
1
1
1
goal
0
0
1
2
match
0
0
1
1
13 / 24
2.5
football (0,2)
goal
1.5
soccer (0,1)
0.5
automobile (1,0)
car (2,0)
0
0
0.5
1.5
2.5
transport
Distributional Semantics
14 / 24
Computing similarity
automobile
car
soccer
football
wheel
1
1
0
0
transport
1
2
0
0
passenger
1
1
0
1
tournament
0
0
1
1
London
0
1
1
1
goal
0
0
1
2
match
0
0
1
1
car . soccer = 1
car . football = 2
soccer . football = 5
Distributional Semantics
15 / 24
Distributional Semantics
16 / 24
One-hot representation
Distributional Semantics
16 / 24
Distributional Semantics
17 / 24
Distributional Semantics
17 / 24
Distributional Semantics
17 / 24
Distributional Semantics
18 / 24
Distributional Semantics
18 / 24
Distributional Semantics
19 / 24
General Questions
How do the rows (words, ...) relate to each other?
How do the columns (contexts, documents, ...) relate to each other?
Distributional Semantics
19 / 24
Distributional Semantics
20 / 24
Distributional Semantics
21 / 24
Distributional Semantics
22 / 24
Words as contexts
Parameters
Window size
Window shape - rectangular/triangular/other
Distributional Semantics
23 / 24
Words as contexts
Parameters
Window size
Window shape - rectangular/triangular/other
Distributional Semantics
23 / 24
Words as contexts
Parameters
Window size
Window shape - rectangular/triangular/other
Distributional Semantics
23 / 24
Words as contexts
Parameters
Window size
Window shape - rectangular/triangular/other
Distributional Semantics
23 / 24
Distributional Semantics
24 / 24
1
|Di |
Distributional Semantics
24 / 24
1
|Di |
Distributional Semantics
24 / 24