Bayesian Adaptive User

Profiling with Explicit & Implicit

Feedback
Philip Zigoris, Yi Zhang
University of California, Santa Cruz
 Obstacles to Personalization

1. Asking for feedback is intrusive.

Implicit Feedback - Infer user feedback
from user behavior.

• Little to no information about new

users (the “cold-start problem”).
Borrow information from existing users
via Bayesian hierarchical model.
11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 2
 Outline

• Implicit Feedback

• Hierarchical Bayesian Framework

• Gaussian Network

• Experiments

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 3

 Implicit Feedback

A user’s interaction with a document suggests

their opinion of it.
Includes:
– Keyboard usage
– Mouse usage
– Viewing time
– Eye-tracking
Appeal: cheap! *
*Modulo security issues
Question: Does it have any predictive value?

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 4

 Is Implicit Feedback Useful?

• Yes
– Positive correlation between the time spent
viewing a page and a user’s opinion of the
page. [Claypool et al., 2001; Fox et al.,
2005]
• No (sometimes)
– In Kelly et al., 2004 demonstrates that
correlation varies significantly across tasks.

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 5

 Outline

• Implicit Feedback

• Hierarchical Bayesian Framework

• Gaussian Network

• Experiments

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 6

 The Task at Hand
Doc Rating
X1 Y1

f X2
X3
Y2
Y3

I Like!
11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 7
 Cold-Start Problem

• Personalized systems require training

data.
• Users do not want to endure poor
performance while the system is
“learning”.
Solution: Give the system a head-
start with some (Bayesian) prior
knowledge.
11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 8
 Bayesian Prior over User Profiles
User-Model Space

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 9

 Hierarchical User Model
Generic Form

f ~ P( f |q)
u
Parameter describing
prior distribution

y ~ f (x)
u

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 10

 Refining Beliefs Based on User Data

P( f |q,(x
u
|q) 1, y1 ),(x
)) , y 2543 ,))y 534 ))
),K 2,(x
11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 11
 The Posterior Distribution

P(D | f ,q)P( f |q)

u u u
P( f |q,D ) =
u u

P(D |q)
u

Maximum A Posteriori model:

Influence of Data
f u
MAP = prior [ P( f
argmax | f )]
|q)P(Dlikelihood u

= arg ma x [log(P( f |q))  +   log(P(D | f ))] u

f
11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 12
 Tradeoff: Prior vs. Data

More data data likelihood term will

dominate the objective function.

log(P(D | f )) = log(Õ P( f (x i ) = y i ))
u

= å log(P( f (x i ) = y i ))

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 13

 Outline

• Implicit Feedback

• Hierarchical Bayesian Framework

• Gaussian Network

• Experiments

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 14

 Gaussian Network

w u
~ N(m,S)
y ~ N(x ×w ,k )T u 2
u

é 1 ù
w MAP
u
= argminê(w - m)T S- 1 (w - m) + å (x Ti ×w - y i ) 2 ú
w ë ku i û

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 15

 Learning the Prior from
Existing Users

Unbiased estimator for prior parameters:

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 16

 Outline

• Implicit Feedback

• Hierarchical Bayesian Framework

• Gaussian Network

• Experiments

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 17

 Experimental Datasets
Claypool
•75 Student Volunteers, 1823 Documents
•Unguided, unrestricted web browsing for 20-30 minutes
•Every document explicitly rated (Scale 1 to 5)

Zhang
•15 Users, 4663 Documents
•1 month, users spent >1 hour everyday using system
• Focused on reading news articles
•Not every document is explicitly rated (Scale 1 to 5)
•Includes other forms of explicit feedback (TBA)
11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 18
 Feature Sets

Type Examples
•Relevance Score
Explicit(Zhang)
•Readability Score
•Mouse usage
Implicit
•Keyboard usage
•Time spent on page
Document •Document length
•Speed of host server
•Number of pages linking to host server

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 19

 Evaluation Methodology

• Tested four models

– Linear model with (P)rior
– Linear model with (N)o prior
– (G)eneric, user-independent model
– Moving (A)verage
• Original ordering preserved
• Leave-one-user-out
• Model updated and evaluated (error2) after
every example

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 20

 Summary of Results
Implicit & Implicit
Explicit Explicit

Performance (MSE) averaged over time and user.

Hierarchical model significantly*

outperforms other methods.
* (95% Wilcoxon signed rank test)
11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 21
 Performance Over Time
Explicit & Implicit Feedback
Shifting rating bias…

“Personalization” seems
to begin
Prior getting
in the way

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 22

 Performance Over Time
Implicit Feedback Only (Zhang et al)

Follows moving
average

Implicit feedback
becomes useful

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 23

 Performance Over Time
Implicit Feedback Only (Claypool et al)

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 24

 Comparative Performance
with Different Feature Sets

Implicit feedback
hurts performance

Implicit feedback
benefits performance

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 25

 Discussion: The Dynamics of the
User Model

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 26

 Discussion: Is Implicit Feedback
Useful? Why?
Error can be decomposed into:
• Bias - How closely the learning algorithm can approximate the best
solution.
• Variance - Sensitivity of learning algorithm to the training sample.
• Noise - Irreducible uncertainty of problem

Including implicit feedback increases variance. This explains why:

•Negative impact on early performance
•Requires substantial amount of training data to be useful.
Linear model has strong bias. This may explain why:
•Implicit feedback does not perform well on its own.

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 27

 Discussion: Shifting User
Behavior

Implicit assumption of our model:

User behavior is consistent over time.

• Already observed shifting bias in rating.

• Also, observed shifts in keyboard and
mouse usage.

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 28

 Conclusion

• Implicit feedback (of the type used in our study) has

marginal predictive value.
– A question of when, not if
– Requires personalization
– Requires lots of data
• Hierarchical model effectively alleviates cold-
start problem
• Shifting user behavior exists

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 29

 Thank You!

11/8/2006 Bayesian Profiling with Explicit & Implicit Feedback 30