RECOMMENDATION

SYSTEMS
INDEX
Recommendation system
1. Traditional collaborative filtering
2. Cluster models
3. Search-based models

Industry application: Amazon

Impact of recommender system: Amazon

1. Hypothesis
2. Results

Conclusion
Recommendation systems
E-commerce sites today are using various sophisticated methods to best cater to the needs of their
customer in order to ensure stickiness and give impetus to repeat purchase. One of the ways to make
the customers feel special and give them a customized service, not unlike that of a counter
salesperson, who gives recommendations in order to cross-sell or up-sell, is the use of the
Recommendation Algorithms. These algorithms use the previous data regarding customers interests
and use it to make recommendations for future purchases. These recommendations made by the
algorithm are based on a number of factors like items viewed, genre interests, ratings given by the
user, demographics etc. The common methods to approach a recommendation problem are cluster
models, collaborative filtering and search-based methods. The way these algorithms work is by
searching for customer sets that had certain similar purchases and ratings. Then these algorithms
aggregate the purchase items of a segment, then for a particular customer it finds the items from the
aggregated list that the customer hasnt purchased, which are then portrayed at recommendations.
The three approached to recommendation problems have been described below:
Traditional Collaborative Filtering
In this algorithm, a customer is represented as an N-dimensional vector wherein N is the number of
distinct items purchased by the customer. Positive components of the vector are the positively rated
items while negative components of the vector are the negatively rated items. To compensate for the
best-selling items, the items vector is multiplied by the inverse frequency matrix, making the lesser
known items more relevant. The algorithm first finds customers similar to the one in question, the
common way of gauging which is by computing the cosine of the angle between the two vectors. After
that it makes recommendations by ranking the items according to the number of similar customers
who bought them. One major issue with this algorithm is that it is computationally expensive and the
methods to tackle the scaling issues lead to reduction in recommendation quality.
Cluster models
These treat the recommendation problem as a case of classification and segment the entire customer
base on the basis of customer similarity. Then the purchases and ratings are used to give the
recommendations. Using simple clustering techniques similar customers are grouped together. They
start with an initial set of segments to each of which a random customer is allocated. Then all the
other customers are matched with each of these segments and then get allotted to the one with
maximum similarity. These models are better at scalability and performance as compared to the
traditional collaborative filtering as the comparison of each customer is with a specified number of
segments instead of the entire customer data set. Again as with the collaborative filtering the
recommendations produced are not always pertinent as the segments might not contain the most
similar customers.
Search-based models
This is a content based method to base recommendations on. Depending on the items purchased
and the ratings given by the customer, the algorithm finds other items with the same attributed, for
example the same author, director, genre, colour etc. For a user having a smaller number of
purchases the recommendations formed are quite pertinent; however with a customer having a large
set of diverse purchases the quality of recommendations gets reduced, becoming very generic and
broad, as the method will use only a subset of those purchases. They also become very repetitive
and there is no novelty about them.
Industry application: Amazon
The recommendation system used by Amazon is based on an algorithm named Item-to-item
collaborative filtering. Unlike the models discussed above, the item-to-item collaborative filtering
method produces real-time recommendations even with massive data sets while maintaining high
quality. This method finds similar items instead of finding similar customers. Based on the purchases
and ratings given by the users, the algorithm aggregates the similar purchases and then makes
recommendations.
Being a large retailer with millions of customers and thousands of categories, Amazons need for a
powerful recommendation system couldnt be satiated by the existing models. Hence they went about
making one for themselves, which they named the item-to-item collaborative filtering. It is quite similar
to the content based method as it concentrates on finding similar items instead of finding similar
customers. This algorithm matches the items purchased or rated by the customer with other items
and compiles a recommendation list. The similarity can be determined in a number of ways, for
example with the help of user ratings, by product attributes, analyzing the customers baskets for
items that were purchased together. The algorithm basically scans the products bought by a customer
and identifies the other products bought along with that, and then it gauges the similarity between
those pairs only. Whenever a customer goes to search for a product on Amazon, there is a section on
the webpage Customers who bought this also bought which gives a set of recommendations based
on what people also bought along with the item being currently pursued by the customer. One
approach to compute the similarity between a product and other related products is:
For each item in product catalog I1
For each customer C who purchased I1
For each item I2 purchased by customer C
Record that a customer purchased I1 and I2
For each item I2
Compute the similarity between I1 and I2
The similarity can be calculated by using the cosine function as used in the traditional collaborative
filtering, the only difference is that here the vector represents not a customer but an item. This method
is scalable because it does the computation for similar items offline. While it is looking for peoples
purchases and ratings online, it doesnt depend on the number of customers or the number of
categories, rather only on the number of titles the user has purchased and rated. Hence this algorithm
performs well with large as well as limited data sets without compromising on the quality of
recommendations.
Impact of recommender system: Amazon
Analysis of movies sold on a top retailers website has been done to determine the impact of
Amazons Recommender System on Sales Volume and Diversity. To measure the diversity of product
sales, different types of movies genres purchased in addition to individual movies to analyze genre-
level cross-pollination.
Customer visiting a website will be stochastically assigned to a control or treatment group. The
different recommendations are shown for a particular group. The control group is shown nothing.
Following are the variables of interest for each group:
First order impact - Impact on sales volume
1. Individual item view volume: Counts how many items individuals have clicked and viewed.
2. Individual item purchase volume: Counts how many items individuals buy.
Second order impact - Impact on sales diversity
1. Aggregate Firm-level Item Sales Diversity: This measures product sales diversity at the firm
level, with the Gini coefficient.
2. Individual Item Sales Diversity: This measures how the recommenders influence the diversity
of products individuals purchase. Gini coefficient is used in this too.
Hypotheses
Sales volume hypotheses
1. Collaborative filtering recommenders will increase the number of product views compared to no
recommendation condition
2. Collaborative filtering recommenders will increase the number of products purchases compared to
no recommendation condition
Sales Diversity hypotheses:
1. Collaborative filtering recommenders will decrease the aggregate product sales diversity compared
to no recommendation condition i.e. Aggregate Gini (CF) > Aggregate Gini (Control)
2. Collaborative filtering recommenders will increase the individual product purchase diversity
compared to no recommendation condition (control). i.e. average individual Gini (CF)< Average
Individual Gini (Control)
3. Recently viewed recommenders will decrease both the individual product purchase diversity and
the aggregate product sales diversity compared to no recommendation condition. i.e. Average
Individual Gini (Recently viewed) > Average Individual Gini (Control)
Dataset has been taken from a field experiment on a website of one of the top two retailers in North
America. Customers viewing logs and purchase logs have been analysed to study the impact of
recommenders on product purchase volume and diversity at the movie item and genre level.
Results
Results on the 1st order influences on the recommenders are clear. Purchases-based exposes users
to more items and increases the number of items bought, ultimately increasing sales volume.
Quantitatively, Purchased-based CF because a 25% lift in views and a 35% in lift in the number of
items purchased compared to the control group. In contrast to that, View-based items show only 3%
lift in views and a 9% lift in the number of items purchased (not statistically significant). Recently
viewed group show a 5% decrease in view and 12% decrease in the number of items purchased.
Wallet size analysis was not statistically significant suggesting that CFs increase the amount spent by
consumers.
For second order impact of recommenders, CF causes individuals to discover more variety of
products but push each individual to the same set of popular titles, leading to concentration bias at
the aggregate level. Recently Viewed recommenders statistically significantly decreases aggregate
sales diversity, it does not seem to influence individual purchase statistically significantly. It shows that
not all collaborative filtering algorithms are the same. There are two basic differences in purchase-
based CF and View-based CF algorithms
1. Algorithms based on purchase might be simply better at performing in delivering the best fit
products
2. Consumers might be more influenced by purchased also purchased signal than the view also
viewed signal
Conclusion
With the advent of big data, recommenders and personalization technologies are fast taking over
every corner of the web. While certain aspects of these systems are argued in the literature, the fact
that these systems will be of utmost importance for retailers and web-service providers cannot be
argued. Yet, we still have much to learn about how different recommender algorithms influence
market and society. From the societal perspective, critics of recommender systems have pointed out
that recommenders can fragment users into specific information consumption bubbles, which can be
harmful in the context of news and politics. From a management perspective, understanding how
different recommenders drive demand is essential for both operational management and profitable
strategy.