Beruflich Dokumente
Kultur Dokumente
Abstract
arXiv:1504.04558v1 [cs.SI] 17 Apr 2015
19
Within Pinboards
18 Across Pinboards
Squared Euclidean Distance
17
16
15
14
13
n
om ents
ns
eN s
s
Ed fts
s
es
on
ay ory
ts
ts
rt
Ce ors
ng
or
or
s
on
W vel
k
ty
og s
Pr y
gy
on
e
O er
sF s
ok
te
Te too
al
es
en Kid
ur
ig
ph
r in
ng
ur
ee
uc
or
Ph doo
ea eau
iti
ec
um
tio
th
a
M cati
ni
hi
lo
hi
uo
m
a
ot
tn
es
at
t
Bo
ct
Cr
ra
Sp
G
om ddi
v
ol His
od
D
Tr
t
br
eD
O
de
as
no
as
ni
tr a
Ta
M
Fi
sE
D
Q
H
te
rB
ut
u
le
od
sF
A
iy
ar
ch
ic
e
hi
us
lth
rs
ot
nc
ai
D
G
us
Fo
rc
en
Ca
I ll
H
id
ie
H
A
M
H
Sc
lm
W
Fi
al. [2011]), Λ is defined as Algorithm 1 User Profiling by Group Constraint Label Prop-
0
agation
Yi,j
Λi,i = max P . (2) Require: X = {x1 , x2 , . . . , xN } a collection of images
j 0
Yi,k
k M : Fine-tuned Imagenet CNN model
G: Correlation matrix between the labels
We can consider the above process as two stages. In the first 1: Predict the categories Y 0 ∈ RN ×K (K is the number of
stage (1 − Λ)W Y t , we use the similarity between the images categories) of X using the trained CNN model M
to propagate the labels at image level. In the second stage, 2: Extract deep features from M for all X.
the group relationship matrix G is employed to further prop- 3: Calculate the similarity matrix W 0 between X using
agate the group relationship to all images, i.e. we multiply kx −x k2
matrix G with the results of (1 − Λ)W Y t . Next, we present Gaussian kernel function W 0 (i, j) = exp(− i2δ2j ),
the convergence analysis of the proposed label propagation where xi and xj are the deep features for image i and j
framework. respectively.
4: Normalize W 0 to get −1 0
PW =0 D W , where D is diagonal
Convergence Analysis matrix with Dii = j Wi,j .
From Eqn. (1), we have the following formula for Y t+1 . 5: Calculate the diagonal matrix Λ according to Eqn.(2).
6: Calculate the affinity matrix between G between different
t
t+1
X i categories3 .
Y t+1 = ((1 − Λ) W ) Y 0 Gt+1 + ((1 − Λ) W ) ΛY 0 Gi 7: Initialize, iteration index t = 0
i=0 8: repeat
(3)
9: Employ Eqn.(1) to update Y t+1 according to Y t
Since we have 0 ≤ λi,j , wi,j , Gi,j < 1 for all i, j, therefore
t+1 0 t+1 10: until Convergence or t reaches the maximum iteration
limt→∞ ((1 − Λ) W ) Y G = 0. It follows that number
t−1 11: Normalize rows of Y t ∈ RN ×K to get Y 0t ∈ RN ×K .
12: return Y 0t .
X i
lim Y t = ((1 − Λ) W ) ΛY 0 Gi . (4)
t→∞
i=0
re
W dd y
O ns s
g
I n s
Q odu rap
e l g
n
en s
C t ctu
Fom tion
D i g t ie
H ek nin
Pr otoors
W ve olo
om ing
M s tio
s
H me ys
Archit ls
Ta ortse
g
o y
Sp en s
Tr chns
es ri
e
Ki str r
c
H lida
d a
i h
G rde
Ar ima
Fi uca
Phtdo
Tettoo
H stor
Ill o
D leb
O her
H alt
G od
um
C rs
group constrained label propagation framework. Note that in
H r
EdY
ai
a
a
u
e
u
e
u
a
e
An
t
1
step 3, we use a Gaussian kernel function to calculate the sim-
s s el y s ts e s ts y rs er s s s or e s ry th ir k g d m n Y n s rs rt re ls
en ingrav logttoopor enc ote uc aph oo th en Kidtionum om daysto eal Ha ee ninFoo Fil atio DI sig ritieCa A ctu ma
i
ite An
ilarity between two images using the deep features. However, 0.9
ch
Ar
one can employ other techniques such as locally linear em-
D leb
0.8
bedding (LLE) [Donoho and Grimes, 2003] to calculate the
e
e
C
similarity between different instances.
uc
0.7
Ed
Eventually, we obtain users’ interests distribution by aggre-
G de
ar
gating the label distribution of the collections of their images. 0.6
G
In other words, we simply sum up the label distribution of
H
interest prediction results. 0.4
us
Ill
0.3
4 Experiments and Evaluations
P ot O
To evaluate the proposed algorithm, we crawl data from Pin- 0.2
Ph
terest according to a randomly chosen user list consisting of 0.1
748 users. Table 2 gives the statistics of our crawled dataset.
W We ech T
This dataset is used to train our own deep convolutional neu-
T
0
0.65
0.6
0.55
14
12
10
CNN 0.5
LP
8
6
GLP Figure 4: Filters of the first convolutional layer. 0.45
4
0.4
4.2
2
0
Experimental Results
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
0.35 CNN
0.3 LP
0.2
GLP
CNN
Proportion of
0.15
Frequencies
0.25
LP
0.1 1 2 3 4 5 6 7 8 9 10
GLP
0.05