Matrix Factorisation

Thierry Silbermann, Data Scientist at Nubank

(and dimensionality reduction)

What is Matrix Factorisation?• For some domain, we have matrix that are:

very ‘big’

sparse

and don’t have any order

• Factoring it yield a set of:

More manageable

compact

and ordered matrices

Matrix Factorisation / Decomposition

• Singular Value Decomposition (SVD)

• Principal Component Analysis (PCA)

• Non-negative Matrix Factorisation (NMF)

• LU/QR/Cholskey decomposition

• etc…

SVD (Singular Value Decomposition)

• The singular value decomposition of an m × n matrix M is a factorisation of the form :

M = UΣV∗,

where U is an m × m unitary matrix,

Σ is an m × n rectangular diagonal matrix with non-negative real numbers on the diagonal (known as the singular values of M),

and V∗ (the transpose of V) is an n × n real or complex unitary matrix.

The m columns of U and the n columns of V are called the left-singular vectors and right-singular vectors of M, respectively.

Limitation SVD• One of the most general algorithm for

decomposition. SVD is an amelioration from eigenvalue decomposition (was only used with n x n matrix)

• Didn’t really reduce the space use to store our data.

• From a m x n (M) matrix, we end up with m x m (U), m x n (Σ) and n x n (V) matrices…

Matrix Factorisation• Using Non-Negative Matrix Factorisation

V ≈ WH

• V, W and H are all non-negative

• V is a n x m matrix

• W is a n x r matrix, H is a r x m matrix

• and r ≪ min(m, n)

How to decompose?

• Minimize with respect to W and H, subject to the constraints W, H ≥ 0.

• Multiplication Update Algorithm

• Alternating Least Square Algorithm

Matrix Factorisation

Toy example

http://www.quuxlabs.com/blog/2010/09/matrix-factorization-a-simple-tutorial-and-implementation-in-python/

Back to the math• We have a matrix of ratings we want to approximate:

• We need to construct P and Q by minimising:

• We have an optimisation problem:

• Update are then give by:

http://www.quuxlabs.com/blog/2010/09/matrix-factorization-a-simple-tutorial-and-implementation-in-python/

Easy to implement (for toy example)

http://www.quuxlabs.com/blog/2010/09/matrix-factorization-a-simple-tutorial-and-implementation-in-python/

Easy to implement

http://www.quuxlabs.com/blog/2010/09/matrix-factorization-a-simple-tutorial-and-implementation-in-python/

Example: Recommend given names to user of the name search engine ”Nameling*”

Example: Recommend given names to user of the name search engine ”Nameling*”

Example: Recommend given names to user of the name search engine ”Nameling*”

*http://nameling.net/

My own advertisement

• libFM (http://www.libfm.org/)

• https://thierrysilbermann.wordpress.com/2015/02/11/simple-libfm-example-part1/

Thank you