Recommender Systems

Chapter 2

Sara Qassimi
[email protected]

L2IS Laboratory , FST Marrakech, Cadi Ayyad University

1
Non-personalized recommender systems
Page Rank
● Used for search results

The PageRank of a page is the probability I would end up on that page

if I surfed the Internet randomly for an infinite amount of times

● Fits into the idea of recommenders: it is jut a score and to make

recommendation like; it sorts items by score
● The higher the score, the higher the page shows up in the search results
● Of Course matching the user’s query search

Pr. Sara Qassimi

FST- UCA 2
Page Rank - Markov Models
● Simplest way to think about markov Models are bigrams from NLP
● Build a probabilistic language model
● can ask “what is the probability the next word in a sentence is ‘do’,
given the previous word was ‘I’?
● p(do | I)
● exemple code: N-gram Modeling With Markov Chains
N_gram_Modeling_Markov.ipynb visualize_MarkovChain.ipynb

Pr. Sara Qassimi 3

FST- UCA
Bigrams
● It is bigram because we only consider 2 words at a time
● Ex. sentence: “I like cats”
● We would not model p(cats | I like) , just p(cats| like)
● Not realistic: What words can come after “and”?
○ Too many to enumerate!
● Does this mean that Markov Models are
Bad?
No! Just results matter

● The result of the model that we actually care about

Pr. Sara Qassimi 4

FST- UCA
 Markov Models
● Each item is a generic state : x(t)
● “Markov”means x(t) does not depend on any values, 2 or more steps behind -
only the immediate last value

● Transition Probability Matrix/ Stochastic Matrix / Markov Matrix :

A(i,j) the probability of going to state j from state i ; its rows must sum to 1
Suppose our states are numbers from 1 to M; then the sum of a single row would be
the sum of over all possible values of J from 1 to M : Probability distribution over J

Pr. Sara Qassimi 5

FST- UCA
Example: Transition Probability Matrix / Stochastic Matrix / Markov Matrix

Suppose we have a system that can be in one of three states: A, B, and C.

These states are represented by numbers 1, 2, and 3.

We want to create a Transition Probability Matrix that describes the probabilities

of transitioning from one state to another.

Transition Probability Matrix P = | 0.3 0.4 0.3 |

| 0.2 0.6 0.2 |
| 0.1 0.2 0.7 |

6
 The probability of staying in state A (P(1,1)) is 0.3 or 30%.
The probability of transitioning from state A to state B (P(1,2)) is 0.4 or 40%.
The probability of transitioning from state A to state C (P(1,3)) is 0.3 or 30%.
For example, if we look at the first row, which corresponds to starting in state A(1):
P(1,1) + P(1,2) + P(1,3) = 0.3 + 0.4 + 0.3 = 1.
This shows that if we are in state A (1), the matrix correctly represents that we will
either stay in A, move to B, or move to C, with probabilities that add up to 1.
Which is the total probability that something will happen, ensuring that the system
will definitely make a transition in the next step.

7
 Markov Chains
Explained Visually
Markov Models : Exemple
● State 1 = Sunny
● State 2 = Rainy
● Let’s suppose:
if today is sunny, then the chance that tomorrow will be sunny is high
The chance that the state goes from sunny to rainy is very small
○ p(sunny | sunny)= 0.9
○ p(sunny | rainy)= 0.1
○ p(rainy | sunny)= 0.1
○ p(rainy | rainy)= 0.9
count up the total number of days it was sunny and count up
the total number of days It became rainy after it was sunny.
Then divide those two numbers to get the probability.

Pr. Sara Qassimi 8

FST- UCA
Calculating probabilities
● What is the probability of the sentence;

● Problem! Suppose we come across a sentence containing a bigram that

did not appear in the training set
The probability is 0! Anything *0 =0

Pr. Sara Qassimi 9

FST- UCA
Calculating probabilities
Solution: Add -1 Smoothing / Add -Epsilon Smoothing
● Add a “fake count’ to every possible bigram
● V= vocabulary size = number of unique words in dataset
● in our case, V= M (number of states), since each state is a word
● E.G;
P(and | and) never occurs bit would get positive probability

Pr. Sara Qassimi 10

FST- UCA
State Distribution
Probability of being in a state at some particular time

Probability of being sunny at time t Probability of being rainy at time t

Pr. Sara Qassimi 11
FST- UCA
Future State Distributions

Pr. Sara Qassimi 12

FST- UCA
Future State Distributions

Given a matrix A, find a vector and a scalar s.t. Multiplying the vector by A is
equivalent to stretching it by the scalar

Pr. Sara Qassimi 13

FST- UCA
 Looking further into the future
What about 2 steps ahead, or more?

This is exactly the kind of model you can build if you want to predict the future
If you can model some process as a Markov model and you can measure your state distribution, then
predicting your state distribution at some point in the future is just a matter of multiplying by A
Pr. Sara Qassimi 14
FST- UCA
PageRank

programmer

Pr. Sara Qassimi 15

FST- UCA
PageRank

Pr. Sara Qassimi 16

FST- UCA
Pr. Sara Qassimi 17
FST- UCA
Non-personalized recommender systems
Popularity
Recommend the most popular items to the users, for instance top-10 movies, top selling
books, the most frequently purchased products.

Pr. Sara Qassimi 18

FST- UCA
Pr. Sara Qassimi 19
FST- UCA
Non-personalized RS : Popularity-based Recommender System

● User cold-start resistant

○ The system can suggest items without any
information about the user.
○ It can be used in environments with a small
number of users.

● Every user has the same recommendation list.

● Item cold start problem:
○ The system cannot propose an item that has
never been selected or a rating by other users
before.

Pr. Sara Qassimi 20

FST- UCA
Pr. Sara Qassimi 21
FST- UCA
What is a good recommendation?

● The one that is personalized (relevant to that user)

● The one that is diverse (includes different user interests)
● The one that doesn’t recommend the same items to users for the
second time
● The one that recommends available products

Pr. Sara Qassimi 22

FST- UCA
Personalized recommender systems
● Personalized recommender system analyzes users data, their
purchases, rating and their relationships with other users in more detail.
In that way every user will get customized recommendations.
● The most popular types are:

Pr. Sara Qassimi 23

FST- UCA
Personalized RS : Content based recommender systems

Recommend items to customer X that are similar to

his/her previous rated items
● The user’s purchase history is observed.

● For example if a user has already read a book from

one author or bought a product of a certain brand it
is assumed that the customer has a preference for
that author or that brand and there is a probability
that user will buy a similar product in the future.

Pr. Sara Qassimi 24

FST- UCA
Similarity Measures

Pr. Sara Qassimi

FST- UCA 25
26
In addition to “users ratings of items“, it uses also “item properties” to build an Recommendation
system that recommend items to customer X that are similar to previous items rated highly by X,
in a 3 macro steps:

Pr. Sara Qassimi 27

FST- UCA
Personalized RS : Content based recommender systems
● No item cold-start problem.
● Recommend items before any users try them.
● Items recommended for one user do not depend on
other users.
● Recommends unpopular products.

● Item information is necessary.

● User cold-start problem
○ The system cannot produce
recommendations when there is not
enough information to build a user profile.

Pr. Sara Qassimi 28

FST- UCA
Content based recommender systems
Disadvantages of content based approach:
● Phenomenon filter bubble: If a user reads a book about a political ideology and books related
to that ideology are recommended to him he will be in the “bubble of his previous interests”.

● Lot of data about user and his preferences needs to be collected to get the best
recommendation

● Sparsity: Insufficient required data to extract descriptive metadata and ratings about items.

● In practice there are 20% of items that attract the attention of 70-80% of users and 70-80% of
items that attract the attention of 20% of users.

● Recommender’s goal is to introduce other products that are not available to users at first
glance. In a content based approach this goal is not achieved as well as in collaborative
filtering.

Pr. Sara Qassimi 29

FST- UCA
 Tutorial
Implementation : Content based- RS

Pr. Sara Qassimi 30

FST- UCA
Tutorial: Building a Simple Content-Based Recommender System

Pr. Sara Qassimi 31

FST- UCA
Tutorial: Content-based Recommender System And Evaluating
Accuracy Metrics

Pr. Sara Qassimi 32

FST- UCA
Tutorial: Visual Content Similarity-based Recommender System

Pr. Sara Qassimi 33

FST- UCA
TP et Rendus

Pr. Sara Qassimi 34

FST- UCA
Exercice 1:

TF-IDF and Cosine Similarity: Implement a content-based recommender system using TF-IDF (Term
Frequency-Inverse Document Frequency) for text features such as product titles and descriptions. Use
cosine similarity to measure similarity between products.

Exercice 2:

Word Embeddings: Instead of TF-IDF, use word embeddings like Word2Vec or GloVe to represent text
features. Compare the results with TF-IDF.

Exercice 3:

Image Classification Model: Build an image classification model (e.g., a convolutional neural network)
to classify products into categories (e.g., men, women, boys, girls). Then, recommend products from
the same category based on image classification.

Exercice 4:

Hybrid Recommender System: Combine both text-based and image-based recommendations. You can
use an ensemble approach or a weighted average of the two recommendation scores.

Pr. Sara Qassimi 35

FST- UCA
Exercice 5:

Matrix Factorization: Implement matrix factorization techniques such as Singular Value

Decomposition (SVD) or Alternating Least Squares (ALS) on the user-product interaction
matrix to make personalized recommendations.

Exercice 6:

Deep Learning Model: Build a deep learning model (e.g., a neural collaborative filtering
model) for content-based recommendations. Use embeddings for both user and product
features and incorporate image and text information.

Exercice 7:

Feature Engineering: Experiment with different features or combinations of features. For

example, try using color information from images or product categories as additional features
for recommendation.

Pr. Sara Qassimi 36

FST- UCA