A TUTORIAL ON NONNEGATIVE MATRIX

FACTORISATION WITH APPLICATIONS TO

AUDIOVISUAL CONTENT ANALYSIS
Slim ESSID & Alexey OZEROV
Telecom ParisTech / Technicolor

July 2014

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

1 / 170

Slides available online

http://www.telecom-paristech.fr/~essid/resources.htm

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

2 / 170

Support
The tutorial is partially supported by the European projects:
FP7 AXES (Access to Audiovisual Archives) http://www.axes-project.eu

FP7 REVERIE (REal and Virtual Engagement in Realistic Immersive

Environments) http://www.reveriefp7.eu/

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

3 / 170

Credits

Some illustrations, slides and demos are reproduced courtesy of:

C. Fvotte,

F. Bimbot,

N. Seichepine,

N. Q. K. Duong,

A. Masurelle,

D. El Badawy,

R. Hennequin,

L. Le Magoarou,

F. Vallet,

L. Chevallier,

A. Liutkus,

J. Sirot,

G. Richard,

V. D. Blondel,

E. Vincent,

L. de Vinci.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

4 / 170

I Introduction
I NMF models
I Algorithms for solving NMF
I Constrained NMF schemes
I Multi-stream and cross-modal NMF schemes
I Applications
I Conclusion

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

5 / 170

Introduction

Motivation

Explaining data by factorisation

General formulation

V(F N)

W(F K ) H(K N)

vn
vn

PK

Essid & Ozerov (TPT/Technicolor)

wk

k=1 hkn wk
A tutorial on NMF

Illustration by C. Fvotte
ICME 2014

6 / 170

Introduction

Motivation

Explaining data by factorisation

General formulation

V(F N)

vn
data matrix

W(F K ) H(K N)

wk

explanatory variables
regressors,
basis, dictionary,
activation coecients,
patterns, topics
expansion coecients

Illustration by C. Fvotte
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

6 / 170

Introduction

Motivation

Principal Component Analysis (PCA)

Recalling the technique1

Assuming the data is real-valued (vn 2 RF ) and centered (E[v] = 0),

PCA returns a dictionary WPCA 2 RF K such that the least squares

error is minimized:
1 X
1
WPCA = min
kvn vn k22 = kV WWT Vk2F
W N
N
n

A solution is given by:

WPCA = E1:K
where E1:K denotes the K dominant eigenvectors of Cv :
Cv = E[vvT ]
1

1 X
vn vn .
N n

slide adapted from (Fvotte, 2012).

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

7 / 170

Introduction

Motivation

Explaining face images by PCA2

Image example: 49 images among 2429 from MITs CBCL face dataset

slide adapted from (Fvotte, 2012).

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

8 / 170

Introduction

Motivation

Explaining face images by PCA

Method

Facial
features

Vectorised images

Importance of features
in each image

...

...

...

...
V

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

9 / 170

Introduction

Motivation

Explaining face images by PCA3

Eigenfaces

Red pixels indicate negative values! How to interpret this?

3

slide adapted from (Fvotte, 2012).

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

10 / 170

Introduction

Motivation

Data is often nonnegative by nature4

pixel intensities;

amplitude spectra;

occurrence counts;

food or energy consumption;

user scores;

stock market values;

...

For the sake of interpretability of the results, optimal processing of

nonnegative data may call for processing under nonnegativity
constraints.
4

slide adapted from (Fvotte, 2012).

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

11 / 170

Introduction

First look at the model

The Nonnegative Matrix Factorisation model

NMF provides an unsupervised linear representation of the data:

H
V WH;

W = [wfk ] s.t. wfk

and
H = [hkn ] s.t. hkn

0
0.

Illustration by N. Seichepine
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

12 / 170

Introduction

First look at the model

Why nonnegative factors?

Nonnegativity induces sparsity.

Nonnegativity leads to part-based decompositions.

"Atoms energy cancellation" is not allowed: once an atom is selected with

some energy, it cannot be further concealed by other atoms.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

13 / 170

Introduction

First look at the model

NMF outputs
Image example

Illustration by C. Fvotte
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

14 / 170

Introduction

First look at the model

NMF outputs
Audio example

NMF produces part-based representations of the data:

Illustration by R. Hennequin.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

15 / 170

Introduction

First look at the model

History

NMF is more than 30-year old!

previous variants referred to as:

nonnegative rank fatorisation (Jeter and Pye, 1981; Chen, 1984);
positive matrix factorisation (Paatero and Tapper, 1994);

popularized by Lee and Seung (1999) for learning the parts of objects.

Since then, widely used in various research areas for diverse applications.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

16 / 170

Introduction

First look at the model

Notations I

V : the F N data matrix:

F features (rows),
N observations/examples/feature vectors (columns);

vn = (v1n , , vFn )T : the n-th feature vector observation among a

collection of N observations v1 , , vN ;
vn is a column vector in RF+ ; vn is a row vector;
W : the F K dictionary matrix:

wfk is one of its coecients,

wk a dictionary/basis vector among K elements;

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

17 / 170

Introduction

First look at the model

Notations II

H : the K N activation/expansion matrix:

hn : the column vector of activation coecients for observation vn :

vn

K
X

hkn wk ;

k=1

hk: : the row vector of activation coecients relating to basis vector wk .

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

18 / 170

Introduction

General usages and applications

General usages of NMF I

What for?

NMF is a non-supervised data decomposition technique, akin to latent

variable analysis, that can be used for:

feature learning: like Principal Component Analysis (PCA);

learn NMF on training dataset
Vtrain ! dictionary W

Evaluation for face recognition:

exploit W to decompose new test

examples
PK vn :
vn k=1 hkn wk ; hkn 0

Dataset: Olivetti faces, 40 classes

Classifiers: LDA (Linear Discriminant
Analysis)

Cross-validated results:

use hn as feature vector for

example n.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

PCA
ICA

Accuracy
93%
93%

NMF

96%
ICME 2014

19 / 170

Introduction

General usages and applications

General usages of NMF II

What for?

topics recovery:
assume V = [vfn ] is a (scaled) term-document co-occurrence matrix:
vfn is the frequency of occurrences of word mf in document dn ;

Topics

Words

Documents

Topic importance
indicators

V
Essid & Ozerov (TPT/Technicolor)

W
A tutorial on NMF

H
ICME 2014

20 / 170

Introduction

General usages and applications

Topics recovery
NMF link to Probabilistic Latent Semantic Analysis (PLSA)

topics recovery: like Probabilistic Latent Semantic Analysis (PLSA):

assume V = [vfn ] is a (scaled) term-document co-occurrence matrix:
vfn is the frequency of occurrences of word mf in document dn ;

PLSA model (Hofmann, 1999)

P(mf , dn ) =

K
X
k=1

P(tk )P(dn |tk )P(mf |tk )

! the documents can be explained by some underlying topics tk .

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

21 / 170

Introduction

General usages and applications

Topics recovery
NMF link to Probabilistic Latent Semantic Analysis (PLSA)

k )P(m
f |tk ) and hkn = P(d
n |tk );
Let wfk = P(t
the model can be re-written as:

f , dn )] = [
[P(m
vfn ] = WH
The wk can be interpreted as topics explaining the data being analyzed to
the extent given by related hk: .
Link between NMF and PLSA (Gaussier and Goutte, 2005)

Any (local) maximum likelihood solution of PLSA is a solution of NMF

with Kullback-Leibler (KL) divergence.

Any solution of NMF with KL divergence yields a (local) maximum

likelihood solution of PLSA.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

22 / 170

Introduction

General usages and applications

Text document analysis example

After sklearn topics extraction demo (Pedregosa et al., 2011)

Analysing the 20 newsgroups dataset with NMF, the following topics are
automatically determined:

Topic #0: god people bible israel jesus christian true moral think
christians believe don say human israeli church life children jewish

Topic #1: drive windows card drivers video scsi software pc thanks vga
graphics help disk uni dos file ide controller work

Topic #2: game team nhl games ca hockey players bualo edu cc year
play university teams baseball columbia league player toronto

Topic #3: window manager application mit motif size display widget
program xlib windows user color event information use events values

Topic #4: pitt gordon banks cs science pittsburgh univ computer soon
disease edu reply pain health david article medical medicine

Topics described by most frequent words in each dictionary element Wk .

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

23 / 170

Introduction

General usages and applications

General usages of NMF III

What for?

clustering: like K-means (Ding et al., 2005, 2010; Xu et al., 2003):

Cluster
centroids

Data points

Cluster membership
indicators

I NMF can handle overlapping clusters and provides soft cluster

membership indications.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

24 / 170

Introduction

General usages and applications

General usages of NMF IV

What for?

temporal segmentation: like Hidden Markov Models (HMM);

analysing temporal data sequences, e.g., videos:

h1:

+ ... +

Essid & Ozerov (TPT/Technicolor)

hK :

w1

wK

A tutorial on NMF

ICME 2014

25 / 170

Introduction

General usages and applications

General usages of NMF IV

What for?

temporal segmentation: like Hidden Markov Models (HMM);

analysing temporal data sequences, e.g., videos:

h1:

hK :

+ ... +

V
w1
wK
Temporal segmentation can be achieved by thresholding the temporal
activations relating to components of interest.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

25 / 170

Introduction

General usages and applications

General usages of NMF V

What for?

filtering and source separation: as with Independent Component

Analysis (ICA):
Spectral
Temporal
Mixture
activations
spectrogram patterns
frequency

time
V

H = [H1 ; H2 ]

W = [W1 , W2 ]

Source 1 model
H1

=
W1
Essid & Ozerov (TPT/Technicolor)

Source 2 model
H2

+
W2

A tutorial on NMF

ICME 2014

26 / 170

Introduction

General usages and applications

In summary...
What for?

NMF is a non-supervised data decomposition technique, akin to latent

variable analysis, that can be used for:

topics recovery: like Probabilistic Latent Semantic Analysis (PLSA);

feature learning: like Principal Component Analysis (PCA);

clustering: like K-means;

temporal segmentation: like Hidden Markov Models (HMM);

filtering and source separation: as with Independent Component

Analysis (ICA);

coding as with vector quantization.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

27 / 170

Introduction

General usages and applications

Overview of NMF application domains I

A variety of successful applications:
Text mining: (Xu et al., 2003; Berry and Browne, 2006; Kim and Park, 2008)

Images:
unsupervised object discovery (Sivic et al., 2005)
object and face recognition (Soukup and Bajla, 2008)
tagging (Kalayeh et al., 2014)
denoising and inpainting (Mairal et al., 2010)
texture classification (Sandler and Lindenbaum, 2011)
spectral data (Berry et al.)
hashing (Monga and Mihcak, 2007)
watermarking (Lu et al., 2009)

Electroencephalography (EEG) data:

feature extraction (Cichocki and Rutkowski, 2006; Lee et al., 2009)
artifact rejection (Damon et al., 2013a,b)

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

28 / 170

Introduction

General usages and applications

Overview of NMF application domains II

Bioinformatics:
gene expression analysis (Brunet et al., 2004; Gao and Church, 2005)
protein interaction clustering (Greene et al., 2008)

Other:
collaborative filtering (Melville and Sindhwai, 2010)
community discovery (Wang et al., 2010)
portfolio diversification (Drakakis et al., 2007)
food consumption analysis (Zetlaoui et al., 2010)
industrial source apportionment (Limem et al., 2013)

Audio and music

Videos

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

29 / 170

Introduction

General usages and applications

Audio and music processing

Source separation (NMF is state-of-the art):

speech: separating voices in speech mixtures or voice from background
(Virtanen, 2007; Virtanen and Cemgil, 2009; Mohammadiha et al., 2013)
music: separating singing voice/melody from accompaniment or musical
instruments in polyphonic mixtures (Durrieu et al., 2009; Ozerov and Fevotte,
2010; Hennequin et al., 2011; Ozerov et al., 2013; Rafii et al., 2013)

Signal enhancement/denoising:
(Wilson et al., 2008; Schmidt et al., 2007; Sun and Mazumder, 2013)

Audio inpainting
(Roux et al., 2011; Yilmaz et al., 2011)

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

30 / 170

Introduction

General usages and applications

Audio and music processing

Compression
(Ozerov et al., 2011b; Nikunen et al., 2011)

Music transcription: recognizing musical notes played by Piano, Drums

or multiple instruments
(Smaragdis and Brown, 2003; Abdallah and Plumbley, 2004; Vincent et al., 2007; E.
Vincent et al., 2008; Fvotte et al., 2009; Bertin et al., 2010; Vincent et al., 2010)

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

31 / 170

Introduction

General usages and applications

Video processing

NMF use for video processing remains quite limited, despite its potential.

Known works:
Video summarization (Cooper and Foote, 2002)
Dynamic video content representation and scene change detection (Bucak and
Gunsel, 2007)

Onscreen person spotting and shot-type classification (Essid and Fevotte, 2012,
2013)

Fingerprinting (Cirakman et al., 2010)

Action recognition (Krausz and Bauckhage, 2010; Masurelle et al., 2014)
Compression (Trkan and Guillemot, 2011)

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

32 / 170

Introduction

General usages and applications

Action recognition using depth silhouettes

Using NMF for feature learning (Masurelle et al., 2014)
depth-map sequence
background
subtraction

Skeleton
features

depth silhouettes

78%

cropping
and resizing
invariant depth silhouettes

NMF
decomposition
local spatio-temporal features
HMM
classification

NMF

89%

91%

Recognition accuracies

temporal
integration
motion-trace depth silhouettes

PCA

considering Kinect recordings

of 8 actions;

using Huawei/3DLife grand

challenge dataset for action
recognition.

recognized gesture
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

33 / 170

Introduction

General usages and applications

Video Structuring
Using NMF for temporal segmentation and soft-clustering (Essid and Fevotte, 2013)

Discovering the video editing

structure (Essid and Fevotte, 2012)
"Full group"

Performing speaker diarization

(Seichepine et al., 2013)

"Multiple participants" "Multiple participants"

Who spoke when?

"Participant 1"

"Participant 3"

"Participant 2"

"Participant 2"

"Participant 4"

"Participant 5"

illustration by N. Seichepine

Using the Canal9 political debates database (Vinciarelli et al., 2009).

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

34 / 170

Introduction

Diculties in NMF

I Introduction
Motivation
First look at the model
General usages and applications
Diculties in NMF
I NMF models
I Algorithms for solving NMF
I Constrained NMF schemes
I Multi-stream and cross-modal NMF schemes
I Applications
I Conclusion

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

35 / 170

Introduction

Diculties in NMF

Model order choice

A suitable choice of K is very important

Model order K corresponds to the number of rank-1 matrices within the

approximation
The choice of K results in a compromise between
Data fitting
A greater K leads to a better data approximation
Model complexity
A smaller K leads to a less complex model (easier to estimate, less
parameters to transmit, etc ...)
A right model order choice is important and it depends on the data V
and on the application.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

36 / 170

Introduction

Diculties in NMF

NMF is ill-posed
The solution is not unique

Given V = WH ; W

WQ

1H

0, H

0; any matrix Q such that:

0
0

H
= (WQ)(Q
provides an alternative factorisation V = W
In particular, Q can be any nonnegative
matrix; e.g., in R3 :
2
0 0
4
Q= 0 3
1 0

1 H).

generalised permutation
3
2
05
0

This case is not so problematic: merely accounts for scaling and

permutation of basis vectors wk .
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

37 / 170

Introduction

Diculties in NMF

Geometric interpretation and ill-posedness

NMF assumes the data is well described by a simplicial convex cone Cw
generated by the columns of W:

w1
vi

Cw

w2

Cw =

nP

K
k=1

k wk ;

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

38 / 170

Introduction

Diculties in NMF

Geometric interpretation and ill-posedness

NMF assumes the data is well described by a simplicial convex cone Cw
generated by the columns of W:

w1
vi

Cw

w1
vi
w2

Cw =

nP

K
k=1

k wk ;

Essid & Ozerov (TPT/Technicolor)

Cw

w2

A tutorial on NMF

Problem: which Cw ?

ICME 2014

38 / 170

Introduction

Diculties in NMF

Geometric interpretation and ill-posedness

NMF assumes the data is well described by a simplicial convex cone Cw
generated by the columns of W:

w1
vi

Cw

w1
vi
w2

Cw =

nP

K
k=1

k wk ;

Cw

w2

Problem: which Cw ?

! Need to impose constraints on the set of possible solutions to select the

most useful ones.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

38 / 170

Introduction

Diculties in NMF

Constrained NMF methods

Dierent types of constraints have been considered in previous works:
Sparsity constraints: either on W or H (e.g., Hoyer, 2004; Eggert and Korner,
2004);
Shape constraints on wk , e.g.:
I
I

convex NMF: wk are convex combinations of inputs (Ding et al., 2010);

harmonic NMF: wk are mixtures of harmonic spectra (Vincent et al., 2008).

Spatial coherence or temporal constraints on hk : activations are smooth

(Virtanen, 2007; Jia and Qian, 2009; Essid and Fevotte, 2013);
Cross-modal correspondence constraints: factorisations of related
modalities are related, e.g., temporal activations are correlated (Seichepine
et al., 2013; Liu et al., 2013; Yilmaz et al., 2011);
Geometric constraints: e.g., select particular cones Cw (Klingenberg et al.,
2009; Essid, 2012).
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

39 / 170

NMF models

I Introduction
I NMF models
Cost functions
Weighted NMF schemes
I Algorithms for solving NMF
I Constrained NMF schemes
I Multi-stream and cross-modal NMF schemes
I Applications
I Conclusion
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

40 / 170

NMF models

Cost functions

NMF optimization criteria

NMF approximation V WH is usually obtained through:
min D(V|WH) ,

W,H 0

b is a separable matrix divergence:

where D(V|V)
b =
D(V|V)

and d (x|y ) defined for all x, y

F X
N
X
f =1 n=1

0 is a scalar divergence such that:

d (x|y ) is continuous over x and y ;

d (x|y )

d (x|y ) = 0 if and only if x = y .

0 for all x, y

Essid & Ozerov (TPT/Technicolor)

d (vfn |
vfn ) ,

0;

A tutorial on NMF

ICME 2014

41 / 170

NMF models

Cost functions

Popular (scalar) divergences

Euclidean (EUC) distance (Lee and Seung, 1999)
dEUC (x, y ) = (x

y)

Kullback-Leibler (KL) divergence (Lee and Seung, 1999)

x
dKL (x, y ) = x log
x +y
y
Itakura-Saito (IS) divergence (Fvotte et al., 2009)
x
x
dIS (x, y ) =
log
1
y
y

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

42 / 170

NMF models

Cost functions

Convexity properties
Divergence d (x|y )
Convex on x
Convex on y

Essid & Ozerov (TPT/Technicolor)

EUC
yes
yes

A tutorial on NMF

KL
yes
yes

IS
yes
no

ICME 2014

43 / 170

NMF models

Cost functions

Scale invariance properties5

dEUC ( x| y ) =
dKL ( x| y ) =

dEUC (x|y )

dKL (x|y )

dIS ( x| y ) = dIS (x|y )

The IS divergence is scale-invariant ! it provides higher accuracy in the
representation of data with large dynamic range, such as audio spectra.

slide adapted from (Fvotte, 2012).

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

44 / 170

NMF models

Cost functions

Music transcription demo

Demo slide courtesy of C. Fvotte (Fevotte et al., 2009)
#
" # # ## $ !!!!

!!

!!

!
!

!!

!!

!!

(MIDI numbers: 61, 65, 68, 72)

Three representations of the data.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

45 / 170

NMF models

Cost functions

Music transcription demo

Demo slide courtesy of C. Fvotte (Fevotte et al., 2009)

NMF decomposition with K = 8

Dictionary W

Coefficients H

Reconstructed components

K=2

K=1

15000
2
4
6
8
10

0.2

10000

0
5000

0.2

10000

2
4
6
8
10

0.2
0

5000

0.2
0

K=6

K=5

K=4

K=3

6000
2
4
6
8
10

0.2

4000

0
2000

0.2

2
4
6
8
10

8000
6000
4000
2000
0

2
4
6
8
10

2000

2
4
6
8
10

200

0.2
0
0.2

0.2
0

1000

0.2
0

0.2
0

100

0.2
0

K=8

K=7

4
2
4
6
8
10

2
4
6
8
10

0.2
0

0.2
0

2
0.2
0

0.2
50

100

150

200

250

300

350

400

450

500

100

200

300

400

500

600

0.5

1.5

2.5

3
5

x 10

Pitch estimates:
Essid & Ozerov (TPT/Technicolor)

65.0
68.0
61.0
72.0
(True values: 61, 65, 68, 72)
A tutorial on NMF

0
ICME 2014

46 / 170

NMF models

Cost functions

General parametric families of divergences

-divergence (Eguchi and Kano., 2001)

d (x|y ) =

8
>
<

1)

>
:

x +(
1)y
x
x log y x + y
x
log yx 1
y

xy

2 R\{0, 1}
=1
=0

Generalizes IS ( = 0), KL ( = 1) divergences and EUC ( = 2) distance.

-divergence (Cichocki et al., 2006, 2008)
d (x|y ) =

Essid & Ozerov (TPT/Technicolor)

1
x + (1
( 1)

A tutorial on NMF

)y

x y 1

ICME 2014

47 / 170

NMF models

Cost functions

And many others ...

Separable divergences:
Csiszars divergence (generalizes -divergence) (Cichocki et al., 2006)
Bregman divergence (generalizes -divergence) (Bregman, 1967; I. S. Dhillon
and S. Sra, 2005)
-divergence (A. Cichocki et al., 2011)

etc ...

Nonseparable divergences:
-divergence (Fujisawa and Eguchi, 2008)
-(Rnyis) divergence (Devarajan and Ebrahimi, 2005)
etc ...

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

48 / 170

NMF models

Cost functions

Which divergence to choose?

NMF divergence choice depends on the data and on the application.
One can choose the divergence as follows:

by intuition or from some prior knowledge of the application goal

(e.g., NMF is used for predicting the unseen data while minimizing the
mean squared error =) EUC distance) or invariances (e.g., scale
invariance for music analysis with IS divergence) ;

from some probabilistic considerations (presented in the upcoming

section);

optimize the divergence (e.g. from some parametric family) on some

development data within a particular application.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

49 / 170

NMF models

Cost functions

Statistical viewpoint
For many divergences a probabilistic formulation is possible: the divergence
minimization becomes equivalent to a maximum likelihood criterion (Fvotte
et al., 2009; Cemgil, 2009b):
b =
D(V|V)

Examples:

EUC
KL
IS

b
Divergence D(V|V)
P
(vfn vfn )2
f ,n

P
v
vfn log vfn
vfn + vfn
fn
f ,n

P vfn
vfn
log
1
v
v
f ,n

fn

fn

Essid & Ozerov (TPT/Technicolor)

b + const
log p(V|V)
Probability distribution
vfn Gaussian vfn ,

f ,n

vfn Poisson (
vfn )

vfn Exponential v1
fn

A tutorial on NMF

Q
Q

f ,n

p 1
2

b
p.d.f. p(V|V)

(vfn
exp
2
2

v
1
v fn
(vfn +1) fn

1
v
f ,n fn

exp

vfn )2
2

exp ( vfn )

vfn
vfn

ICME 2014

50 / 170

NMF models

Cost functions

Statistical viewpoint
Numerous advantages of a probabilistic NMF formulation:

possibility of using ecient probabilistic inference algorithms such as

the Expectation-Maximization (EM) algorithm (Fvotte et al., 2009) and
the Monte Carlo methods (Cemgil, 2009b; Schmidt et al., 2009);

possibility of introducing various constraints into NMF modeling via

prior distributions (Arngren et al., 2011);

possibility of learning the NMF from partially missing (Roux et al., 2011)
or noisy (Arberet et al., 2012) data;

possibility of combining the NMF with dierent probabilistic models

(Ozerov et al., 2012), e.g., the hidden Markov models (HMMs) (Ozerov et al.,
2009).

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

51 / 170

NMF models

Weighted NMF schemes

Weighted NMF
Conventional NMF optimization criterion (separable divergence case):
min

W,H 0

F X
N
X
f =1 n=1

d (vfn |
vfn ) .

Weighted NMF optimization criterion:

min

W,H 0

F X
N
X
f =1 n=1

bfn d (vfn |
vfn ) ,

where bfn (f = 1, . . . , F , n = 1, . . . , N) are some nonnegative weights

representing the contribution of data point vfn into NMF learning.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

52 / 170

NMF models

Weighted NMF schemes

Weighted NMF application example I

Learning from partial observations (e.g., for image inpainting as in (Mairal
et al., 2010)):
Observed value
bfn = 1

Missing value
bfn = 0

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

53 / 170

NMF models

Weighted NMF schemes

Weighted NMF application example II

Face feature extraction (example and figure from (Blondel et al., 2008)):
Data V

Weights B = {bfn }f ,n

Image-centered weights

Face-centered weights

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

54 / 170

Algorithms for solving NMF

I Introduction
I NMF models
I Algorithms for solving NMF
Preliminaries
Multiplicative update rules
Model order selection, initialization and stopping criteria
I Constrained NMF schemes
I Multi-stream and cross-modal NMF schemes
I Applications
I Conclusion
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

55 / 170

Algorithms for solving NMF

Preliminaries

Optimization diculties

An ecient solution of the NMF optimization problem

def

min D(V|WH) , min C () ; C () = D(V|WH)

W,H 0

def

(where = {W, H} denotes the NMF parameters) must cope with the
following diculties:

the nonnegativity constraints must be taken into account;

no uniqueness of the solution is guaranteed in general;

the optimization problem has usually a multitude of local and global

minima.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

56 / 170

Algorithms for solving NMF

Preliminaries

Alternating optimization strategy

The problem is usually easier to optimize over one matrix (say H) given the
other matrix (say W) is known and fixed.
Indeed, for several divergences D(V|WH) is even convex separately w.r.t.
H and w.r.t. W, but not w.r.t. {W, H}.
For this reason many state-of-the-art NMF optimization algorithms rely on
the following iterative alternating optimization strategy.
Alternating optimization a.k.a block-coordinate descent (one iteration):

update W, given H fixed,

update H, given W fixed.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

57 / 170

Algorithms for solving NMF

Multiplicative update rules

Multiplicative update rules

A heuristic approach introduced by (Lee and Seung, 2001) to solve min C ()
Multiplicative update (MU) rule for H (similarly for W) is defined as:

hkn
hkn rhkn C () / rhkn C () + ,
where

rhkn C () = rhkn C () +

rhkn C ()

and the summands are both nonnegative.

NOTE: The nonnegativity of W and H is guaranteed by construction.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

58 / 170

Algorithms for solving NMF

Multiplicative update rules

MU rules for the -divergence

For example, in the case of the -divergence (generalizing the three
popular divergences) the following decomposition:
ry d (x|y ) =

y 1
| {z }

xy 2
| {z }

[ry d (x|y )]+

[ry d (x|y )]

leads to the following MU rules (in matrix form) (Fvotte et al., 2009):
MU rules for NMF with the -divergence (one iteration):
H

2]

(WH).[

WT (WH).[

2]

(WH).[
(WH)

.[

1]

V
1]

T
V H

HT

Re-normalize W columns and H rows to address scale-invariance (see Fvotte et al. 2009).
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

59 / 170

Algorithms for solving NMF

Multiplicative update rules

Intuitive explanation
We consider for simplicity rh C (h) = r+

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

60 / 170

Algorithms for solving NMF

Multiplicative update rules

Discussion

The only two things guaranteed by this approach:

the newly updated value lies in the direction of partial derivative

decrease;

the newly updated value is always nonnegative.

Nothing more can be guaranteed in general, and all the other algorithm
properties depend on the positive-negative decomposition chosen:

rhkn C () = rhkn C () +

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

rhkn C ()

ICME 2014

61 / 170

Algorithms for solving NMF

Multiplicative update rules

Gradient descent viewpoint

Each MU rule can be interpreted as a diagonally rescaled gradient

descent (Lee and Seung, 2001):
hkn

hkn

kn rhkn C () ,

where the step-size kn is defined as kn = hkn / rhkn C () + .

Though this re-formulation does not bring any new properties for the
algorithm (e.g., the convergence).

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

62 / 170

Algorithms for solving NMF

Multiplicative update rules

Majorisation-minimisation viewpoint
For many divergences and certain positive-negative decompositions
each MU rule can be interpreted as a Majorisation-Minimisation (MM)
procedure (Hunter and Lange, 2004):
To minimise C (s), e.g., s = wfk or s = hkn :

build G (s|s ) such that G (s|s )

C (s) and G (s |s ) = C (s );

optimize iteratively G (s|s ) instead of C (s).

0.5
Objective function C(s)

0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0

Illustration by C. Fvotte
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

63 / 170

Algorithms for solving NMF

Multiplicative update rules

Majorisation-minimisation viewpoint
For many divergences and certain positive-negative decompositions
each MU rule can be interpreted as a Majorisation-Minimisation (MM)
procedure (Hunter and Lange, 2004):
To minimise C (s), e.g., s = wfk or s = hkn :

build G (s|s ) such that G (s|s )

C (s) and G (s |s ) = C (s );

optimize iteratively G (s|s ) instead of C (s).

0.5
Objective function C(s)

0.45

Auxiliary function G(s|s(0))

0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0

Illustration by C. Fvotte
Essid & Ozerov (TPT/Technicolor)

(1)

A tutorial on NMF

(0)

ICME 2014

63 / 170

Algorithms for solving NMF

Multiplicative update rules

Majorisation-minimisation viewpoint
For many divergences and certain positive-negative decompositions
each MU rule can be interpreted as a Majorisation-Minimisation (MM)
procedure (Hunter and Lange, 2004):
To minimise C (s), e.g., s = wfk or s = hkn :

build G (s|s ) such that G (s|s )

C (s) and G (s |s ) = C (s );

optimize iteratively G (s|s ) instead of C (s).

0.5
Objective function C(s)

0.45

Auxiliary function G(s|s(1))

0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0

Illustration by C. Fvotte
Essid & Ozerov (TPT/Technicolor)

(2)

(1)

A tutorial on NMF

(0)

ICME 2014

63 / 170

Algorithms for solving NMF

Multiplicative update rules

Majorisation-minimisation viewpoint
For many divergences and certain positive-negative decompositions
each MU rule can be interpreted as a Majorisation-Minimisation (MM)
procedure (Hunter and Lange, 2004):
To minimise C (s), e.g., s = wfk or s = hkn :

build G (s|s ) such that G (s|s )

C (s) and G (s |s ) = C (s );

optimize iteratively G (s|s ) instead of C (s).

0.5
Objective function C(s)

0.45

Auxiliary function G(s|s(2))

0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0

Illustration by C. Fvotte
Essid & Ozerov (TPT/Technicolor)

(3)

(2)

(1)

A tutorial on NMF

(0)

ICME 2014

63 / 170

Algorithms for solving NMF

Multiplicative update rules

Majorisation-minimisation viewpoint
For many divergences and certain positive-negative decompositions
each MU rule can be interpreted as a Majorisation-Minimisation (MM)
procedure (Hunter and Lange, 2004):
To minimise C (s), e.g., s = wfk or s = hkn :

build G (s|s ) such that G (s|s )

C (s) and G (s |s ) = C (s );

optimize iteratively G (s|s ) instead of C (s).

0.5
Objective function C(s)

0.45

Auxiliary function G(s|s*)

0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0

Illustration by C. Fvotte
Essid & Ozerov (TPT/Technicolor)

(3)

(2)

(1)

A tutorial on NMF

(0)

ICME 2014

63 / 170

Algorithms for solving NMF

Multiplicative update rules

Majorisation-minimisation viewpoint
For many divergences and certain positive-negative decompositions
each MU rule can be interpreted as a Majorisation-Minimisation (MM)
procedure (Hunter and Lange, 2004):
To minimise C (s), e.g., s = wfk or s = hkn :

build G (s|s ) such that G (s|s )

C (s) and G (s |s ) = C (s );

optimize iteratively G (s|s ) instead of C (s).

I NOTE: The MM procedure guarantees the cost is non-increasing at

each iteration:
C (s (t+1) ) G (s (t+1) |s (t) ) G (s (t) |s (t) ) = C (s (t) ).

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

63 / 170

Algorithms for solving NMF

Multiplicative update rules

Convergence analysis
Monotonicity (convergence in terms of non-increase of the cost):

is not guaranteed in general for MU rules;

is proven (via the majorisation-minimisation formulation) for some

divergences (e.g., and -divergences) with particular positive-negative
decompositions (see, e.g., Fvotte and Idier 2010; Yang and Oja 2011).

Local convergence in parameters (whether the solution converges to a

stationary point?)

very few positive results for MU rules (see, e.g., Lin 2007a; Badeau et al.
2010);

the main diculty is due to non-uniqueness of the NMF.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

64 / 170

Algorithms for solving NMF

Multiplicative update rules

Summary

Advantages:

easy to implement;

non-negativity of W and H is guaranteed.

Drawbacks:

monotonicity is not always guaranteed;

among other algorithms the convergence rate is not the highest one.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

65 / 170

Algorithms for solving NMF

Multiplicative update rules

Other alternating optimization algorithms

Gradient-like algorithms (Lin, 2007b)
Advantages: may converge faster than MU rules
Drawbacks: nonnegativity constraints must be explicitly handled.
Newton-like algorithms (Zdunek and Cichocki, 2006)
Advantages: converge faster than Gradient-like algos and MU rules
Drawbacks: nonnegativity constraints must be explicitly handled;
limited to convex divergences
Expectation-maximization (EM) algorithms (Fvotte et al., 2009; Cemgil,
2009a)

Advantages: nonnegativity constraints are implicitly handled;

possibility of introducing other constraints via probabilistic priors
Drawbacks: may converge slower than MU rules; limited to NMF
with probabilistic formulation

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

66 / 170

Algorithms for solving NMF

Multiplicative update rules

Online algorithms

Online algorithms to handle continuous data streams (Bucak and Gunsel,

2009; Simon and Vincent, 2012)

Online algorithms to handle big data (stochastic gradient-like) (Mairal

et al., 2010)

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

67 / 170

Algorithms for solving NMF

Model order selection, initialization and stopping criteria

How to choose model order?

A right model order choice is important and it depends on the data V
and on the application.
The following strategies are usually used to set up an appropriate model
order:

Model order K is fixed during the NMF decomposition, and it was

either chosen by intuition,
either chosen based on some prior knowledge (e.g., known number of clusters
for clustering),
or trained on some development data within a particular application.

Model order K is estimated automatically within the NMF

decomposition (Tan and Fvotte, 2013; Schmidt and Morup, 2010).

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

68 / 170

Algorithms for solving NMF

Model order selection, initialization and stopping criteria

Model order choice

Illustration on audio data

WH

K =1

K = 12

K = 24

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

69 / 170

Algorithms for solving NMF

Model order selection, initialization and stopping criteria

Initialization
A good initialization of parameters (W and H) is important for any
local optimization approach (including MU rules) due to the existence of
many local minima.
Random initializations:

initialize (nonnegative) parameters randomly several times;

keep the solution with the lowest final cost.

Strcutural data-driven initializations:

initialize W by clustering of data points V (Kim and Choi, 2007);

initialize W by singular value decomposition (SVD) of data points V

(Boutsidis and Gallopoulos, 2008);

etc ...

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

70 / 170

Algorithms for solving NMF

Model order selection, initialization and stopping criteria

Stopping criteria
How many iterations?

For any iterative optimization strategy (including MU rules) the total

number of iterations is important and results in a tradeo between:

the computational load from one side, and

the data fitting (approximation error) and model quality from the other
side.

Stopping criteria (Albright et al., 2006):

after a fixed number of iterations;

once the approximation error (the cost) is below a pre-defined

threshold;

once the approximation error relative decrease is below a pre-defined

threshold;

etc ...

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

71 / 170

Constrained NMF schemes

I Introduction
I NMF models
I Algorithms for solving NMF
I Constrained NMF schemes
Regularized NMF
Geometric approaches
I Multi-stream and cross-modal NMF schemes
I Applications
I Conclusion
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

72 / 170

Constrained NMF schemes

Regularized NMF

Motivation
Reminder !
Problems:

NMF is not unique.

Hence NMF is not guaranteed to extract latent components as

desired within a particular application.

Possible solution: Given the application, impose some knowledge-based

constraints on W, on H, or on both W and H.

Adding constraints usually makes the decomposition more unique.

Appropriate constrains may lead to more suitable latent components.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

73 / 170

Constrained NMF schemes

Regularized NMF

Shape-constrained NMF
Convex NMF (Ding et al., 2010)

Constrain the basis vectors wk to be convex combinations of the input

vectors:
Convex-NMF model
wk =

N
X

gnk vn ; gnk

0,

gnk = 1

n=1

hence the model:

V (VG)H ; hkn

I Remarks:

vectors wk can be better interpreted as centroids, being weighted sums of

data points;
the solution (G, H) tends to be more sparse (Ding et al., 2010);
in practice, the model yields more regular solutions than NMF (see, e.g.,
Essid and Fevotte 2013).

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

74 / 170

Constrained NMF schemes

Regularized NMF

Shape-constrained NMF
Harmonic NMF (Vincent et al., 2008)

Many audio sound (e.g., speech, harmonic music sounds, etc.) exhibit a
harmonic structure.

Frequency

Partials

Fundemental
frequency (f0)
Time

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

75 / 170

Constrained NMF schemes

Regularized NMF

Shape-constrained NMF
Harmonic NMF (Vincent et al., 2008)

Constrain the basis vectors wk to be mixtures of M pre-defined

M
narrow-band harmonic spectra E = {em }m=1 .
Harmonic NMF model
wk =

M
X

gmk em ; gmk

0,

m=1

where many entries of matrix G = {gmk }m,k are constrained to be zero

(combining any harmonic spectra together is not allowed).
Hence the model:
V (EG)H ; hkn 0 ; gkn 0.
I Remarks:
resulting W = EG is always harmonic by construction;
(G, H) includes less free parameters than (W, H) in the unconstrained NMF.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

76 / 170

Constrained NMF schemes

Regularized NMF

Shape-constrained NMF
Example of harmonic NMF (Ozerov et al., 2012)

A slightly modified version of a figure from (Ozerov et al., 2012).

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

77 / 170

Constrained NMF schemes

Regularized NMF

Sparse NMF
Sparsity constraints on H

Enforcing only few non-zero entries in hn :

Sparse H

Unconstrained H

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

78 / 170

Constrained NMF schemes

Regularized NMF

Sparse NMF
Group sparsity constraints on H

Enforcing only few non-zero pre-defined groups (blocks) in H:

Group sparse H

Unconstrained H

Essid & Ozerov (TPT/Technicolor)

Group =
"2 rows"
W

A tutorial on NMF

ICME 2014

79 / 170

Constrained NMF schemes

Regularized NMF

Sparse NMF
Implementation

Sparsity and group sparsity constraints on H and/or W are usually

implemented by adding sparsity-inducing penalties to the divergence to
be minimized:
C (W, H) = D(V|WH) +

(H) + (W) .

See for example:

(Hoyer, 2004; Eggert and Korner, 2004) for sparsity-inducing penalties in

(A. Lefvre et al., 2011; Sun and Mazumder, 2013; El Badawy et al., 2014) for
group sparsity-inducing penalties in NMF.

NMF,
and

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

80 / 170

Constrained NMF schemes

Regularized NMF

Smooth NMF schemes

Motivation: NMF temporal activations (rows of H) are often erratic,
while we know that for some decompositions they should be rather smooth
(e.g., for music notes activations).
Solution: introducing smoothness constraints into NMF decomposition
(Virtanen, 2007; Jia and Qian, 2009; Essid and Fevotte, 2013; Seichepine et al., 2014a),

e.g., by adding a smoothness penalty to the divergence to be minimized:

C (W, H) = D(V|WH) +

s S(H) ;

where
K

1 XX
S(H) =
|hkn
2
k=1 n=2

hk(n

1) |

p = 1 or 2

is a regularization parameter controlling the amount of smoothing.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

81 / 170

Constrained NMF schemes

Regularized NMF

Smooth NMF schemes

Illustration on synthetic sequential data

Illustration by N. Seichepine

(Seichepine et al., 2014a)

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

82 / 170

Constrained NMF schemes

Regularized NMF

Other constraints
as mentioned in (Gillis, 2014)

graph regularized NMF (Cai et al., 2011),

orthogonal NMF (Choi, 2008),

tri-NMF (Ding et al., 2006),

projective NMF (Yang and Oja, 2010),

minimum volume NMF (Miao and Qi, 2007),

hierarchical NMF (Li et al., 2013),

etc ...

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

83 / 170

Constrained NMF schemes

Geometric approaches

I Introduction
I NMF models
I Algorithms for solving NMF
I Constrained NMF schemes
Regularized NMF
Geometric approaches
I Multi-stream and cross-modal NMF schemes
I Applications
I Conclusion
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

84 / 170

Constrained NMF schemes

Geometric approaches

Preliminaries

w1
vi

Cw

w2

Problem: which Cw ?

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

85 / 170

Constrained NMF schemes

Geometric approaches

Preliminaries

w1
vi

w1

Cw

vi

Cc
W

w2

w2

Problem: which Cw ?

Essid & Ozerov (TPT/Technicolor)

Seek the smallest cone Cc

W
containing the data.
A tutorial on NMF

ICME 2014

85 / 170

Constrained NMF schemes

Geometric approaches

Why the smallest cone?

Assume the data has actually been generated as V = BC; B

C 0;

the exact NMF model is then W = B and H = C.

Klingenberg et al.

0 and

lemma

If Prob{ckn 2 V(0+ )} =
6 0,
i.e. the distribution of the activation coecients ckn is non-zero in a
positive neighborhood of the origin, so that some observations may be
arbitrarily close to the vertices of the generating cone,
c
then the smallest cone Cc
W is exactly the generating cone CB as N ! 1.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

86 / 170

Constrained NMF schemes

Geometric approaches

Why the smallest cone?

Example after (Klingenberg et al., 2009)

Generate data according to V = BC; such that B

uniformly drawn in [0, 1]:

0, C

0 and ckn

Illustration extracted from (Klingenberg et al., 2009)

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

87 / 170

Constrained NMF schemes

Geometric approaches

Advantages

W is uniquely determined (upto a permutation matrix): its columns are

the vertices of the cone Cc
W.
H can then be uniquely determined using standard nonnegative linear
regression.

! the model becomes identifiable!

Simpler algorithms can potentially be devised, significantly lowering the

computational load...

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

88 / 170

Constrained NMF schemes

Geometric approaches

Determining Cc
W
Preliminary

Assume (without loss of generality) that the data is scaled to unit length,
i.e. ||vn || = 1, 8n:

w1
vi

Cc
W

w2

Original data
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

89 / 170

Constrained NMF schemes

Geometric approaches

Determining Cc
W
Preliminary

Assume (without loss of generality) that the data is scaled to unit length,
i.e. ||vn || = 1, 8n:

w1
vi

Cc
W

w1
vi

w2
Cc
W

Original data
Essid & Ozerov (TPT/Technicolor)

w2

Data rescaled to unit-length

A tutorial on NMF

ICME 2014

89 / 170

Constrained NMF schemes

Geometric approaches

2
Determining Cc
W with the EVA in R

EVA: Extreme Vector Algorithm (Klingenberg et al., 2009)

Find the two data vectors w1 and w2

which are the furthest apart in an
angular sense:

w1
vi
Cc
W

Essid & Ozerov (TPT/Technicolor)

w1 , w2 = argmax cos
m,n

w2

T
(vm
vn )

T
= argmin vm
vn
m,n

A tutorial on NMF

ICME 2014

90 / 170

Constrained NMF schemes

Geometric approaches

The EVA in higher dimensions

(Klingenberg et al., 2009)

Initialisation
T v : first two vectors furthest apart
Set w1 , w2 = argminm,n vm
n

For

i = 2 : K (repeat until target rank K is reached)

- Set Wi = w1

...

wi

- Let Pi = Wi (WiT Wi )

WiT : projection onto span{w1 , . . . , wi }

- Project V onto currentspan of Wi

: V0 = Pi v1 . . . vn

- Find k such that vkT vk0 = vkT (Pi vk ) = minn vnT vn0
: the furthest in angular sense to its projection onto
span{w1 , . . . , wi }.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

91 / 170

Constrained NMF schemes

Geometric approaches

Discussion

The basis vectors wk are assumed to be among the data:

V is assumed to be K -separable, that is:

9 K : a set of indices; |K| = K ; such that W = V(:, K)

9 K such that V = V(:, K)H ; H

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

92 / 170

Constrained NMF schemes

Geometric approaches

Discussion

The basis vectors wk are assumed to be among the data:

V is assumed to be K -separable, that is:

9 K : a set of indices; |K| = K ; such that W = V(:, K)

9 K such that V = V(:, K)H ; H 0

in other words, V satisfies the Extreme Data Property (EDP) (Klingenberg
et al., 2009):

Illustration from (Klingenberg et al., 2009)

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

EDP satisfied
ICME 2014

92 / 170

Constrained NMF schemes

Geometric approaches

Discussion

The basis vectors wk are assumed to be among the data:

V is assumed to be K -separable, that is:

9 K : a set of indices; |K| = K ; such that W = V(:, K)

9 K such that V = V(:, K)H ; H 0

in other words, V satisfies the Extreme Data Property (EDP) (Klingenberg
et al., 2009):

Illustration from (Klingenberg et al., 2009)

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

EDP satisfied
ICME 2014

92 / 170

Constrained NMF schemes

Geometric approaches

Discussion

The basis vectors wk are assumed to be among the data:

V is assumed to be K -separable, that is:

9 K : a set of indices; |K| = K ; such that W = V(:, K)

9 K such that V = V(:, K)H ; H

Illustration from (Klingenberg et al., 2009)

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

EDP not satisfied!

ICME 2014

92 / 170

Constrained NMF schemes

Geometric approaches

Discussion

The basis vectors wk are assumed to be among the data:

V is assumed to be K -separable, that is:

The EVA is potentially very complex: its time complexity is O(K 4 )

9 K : a set of indices; |K| = K ; such that W = V(:, K)

9 K such that V = V(:, K)H ; H

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

92 / 170

Constrained NMF schemes

Geometric approaches

Discussion

The basis vectors wk are assumed to be among the data:

V is assumed to be K -separable, that is:

The EVA is potentially very complex: its time complexity is O(K 4 )

9 K : a set of indices; |K| = K ; such that W = V(:, K)

9 K such that V = V(:, K)H ; H

! need for alternative geometric algorithms...

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

92 / 170

Constrained NMF schemes

Geometric approaches

Determining Cc
W by a separating hyperplane
A geometric intuition

w1

vn
P
w2
Cc
W
To determine Cc
W , find the hyperplane P that separates the data from the
origin with maximum margin.

! this is the single-class Support Vector Machine problem!

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

93 / 170

Constrained NMF schemes

Geometric approaches

Single-class Support Vector Machines

Handling outliers

P
margin SV
xi ; ||xi || = 1

||a||

margin
SV

outliers
aka
non-margin SV

P : ha, xi = 0 , 0
a : normal vector; : oset
Essid & Ozerov (TPT/Technicolor)

Single-class SVM problem:

mina,i , 12 ||a||2 +
s.t. ha, xi i

1
N

i , i

i i

,
0,

0;

i 0 : are slack variables (1 i N);

: positive penalization parameter:
I
I

is an upper-bound on the fraction of

outliers;
a lower-bound on the fraction of support
vectors.

A tutorial on NMF

ICME 2014

94 / 170

Constrained NMF schemes

Geometric approaches

Determinig Cc
W using single-class SVM
(Essid, 2012)

P
w1
Cc
W vertices wk are merely the
margin-support vectors.
a

w2
P : ha, xi

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

=0

ICME 2014

95 / 170

Constrained NMF schemes

Geometric approaches

The SVM-NMF algorithm I

(Essid, 2012)

1. Apply single-class SVM on data V = {v1 , , vn };

2. Select margin support vectors as basis vectors wk ;
3. Solve for H: minhi C (hi ) = ||vi

Whi ||2 s.t. hk,i

0;

a classic nonnegative least-squares problem.

I Advantages:
The proposed algorithm can be straightforwardly kernelized, hence:
allowing for non-linear data decompositions;
incorporating prior knowledge through the use of appropriate kernels.

Model order selection: the choice of K is no longer required, it is

determined from the data through a proper choice of .

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

96 / 170

Constrained NMF schemes

Geometric approaches

The SVM-NMF algorithm II

(Essid, 2012)

I Advantages:

Lower complexity compared to reference geometric approach (Extreme

Vector Algorithm) which is O(K 4 ) while SVM can be solved with O(n).

Straightforward adaptation to online processing (online SVM

techniques).

Ability to exclude outliers.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

97 / 170

Constrained NMF schemes

Geometric approaches

Image analysis example

Data generation
1500 images of size 32 32;
generated as positive linear combinations of
5

10

10

10

15

15

15

20

20

20

25

25

25

30

30
5

10

15

20

25

30
5

30

10

15

20

25

30

10

15

20

25

30

Observation example:
5

10

15

20

25

30
5

10

15

20

25

Images as 1024-coecient vectors

Basis vectors stacked in a 1024 3 matrix B

Data generated as V = BC

C drawn uniformly in the range [0, 1]

30

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

98 / 170

Constrained NMF schemes

Geometric approaches

Image analysis example

Components wk found by applying SVM-NMF on V = [BC, B]; = 0.001:
5

10

10

10

15

15

15

20

20

20

25

25

25

30

30
5

10

15

20

25

30
5

30

10

15

20

25

30

10

15

20

25

30

Components wk found by applying SVM-NMF on V = BC; = 0.001:

5

10

10

10

15

15

15

20

20

20

25

25

25

30

30
5

10

15

20

25

30

Essid & Ozerov (TPT/Technicolor)

30
5

10

15

20

25

A tutorial on NMF

30

10

15

20

ICME 2014

25

30

99 / 170

Constrained NMF schemes

Geometric approaches

Image analysis example

Number of support vectors and basis vectors (i.e. margin support
vectors) as a function of N.

! When N increases, outliers are created but the number of components

remains fixed.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

100 / 170

Constrained NMF schemes

Geometric approaches

Synthetic music transcription example

I Data generation
P
s(t) =

m sm (t)rm (t) + b(t)

P10 am
sm (t) = p=1 p cos(2pfm t)

b(t) : Gaussian noise; SNR = 6dB

rm (t) defines the temporal activations
V formed by stacking short-term power spectra column-wise.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

101 / 170

Constrained NMF schemes

Geometric approaches

Synthetic music transcription example

1
0.5
0
1

0.5

1.5

2.5

3.5

4.5

0.5

1.5

2.5

3.5

4.5

0.5

1.5

2.5

3.5

4.5

0.5

1.5

2.5

3.5

4.5

0.5
0
1
0.5
0
1
0.5
0

Estimated activations (in red) allow for a perfect transcription.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

102 / 170

Constrained NMF schemes

Geometric approaches

Non-linear NMF using kernels

: V ! H; H: a feature space.

(x, y ) = h (x), (y)i, (x, y) 2 RF RF .

I Kernel-based NMF problem:

(V) WH s.t. wfk

0 and hkn

I Solution:

Determine W using kernel single-class SVM;

Solve: minhn C (hn ) = 12 || (vn )

Whn ||2H

It is easily shown, using the kernel trick, that:

PK
@C (hn )
(vn , vl ) and
k=1 hkn (vk , vl )
@hkn =

s.t. hkn
@ 2 C (hn )
@hln @hkn

0.
= (vk , vl )

! The Hessian matrix is exactly the Gram matrix which is positive

definite for positive definite kernels.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

103 / 170

Constrained NMF schemes

Geometric approaches

Ongoing work

Considering real-world applications.

Using application-specific kernels: potential for incorporating prior

knowledge on the data.

Studying the impact of on model-order selection.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

104 / 170

Multi-stream and cross-modal NMF schemes

I Introduction
I NMF models
I Algorithms for solving NMF
I Constrained NMF schemes
I Multi-stream and cross-modal NMF schemes
NTF models
Co-factorisation schemes
I Applications
I Conclusion
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

105 / 170

Multi-stream and cross-modal NMF schemes

NTF models

Multi-way data representations

Some data can have more meaningful representation using multi-way
arrays rather than matrices (two-way arrays).
Multichannel audio (Fvotte and
Ozerov, 2010)

Text statistics for natural language

processing (de Cruys, 2010)
verb

subject

frequency

channel

time

Essid & Ozerov (TPT/Technicolor)

object

A tutorial on NMF

ICME 2014

106 / 170

Multi-stream and cross-modal NMF schemes

NTF models

Multi-way data representations

Some data can have more meaningful representation using multi-way
arrays rather than matrices (two-way arrays).
Electroencephalography (EEG) data (Lee et al., 2007)
channel
V

frequency

frequency

channel

time

Essid & Ozerov (TPT/Technicolor)

time

A tutorial on NMF

ICME 2014

107 / 170

Multi-stream and cross-modal NMF schemes

NTF models

Multi-way data representations

Some data can have more meaningful representation using multi-way
arrays rather than matrices (two-way arrays).
Tag recommendation (Rendle et al.,
2009)

Context-aware collaborative
filtering (Karatzoglou et al., 2010)
user

context

tag

user

item

Essid & Ozerov (TPT/Technicolor)

movie

A tutorial on NMF

ICME 2014

108 / 170

Multi-stream and cross-modal NMF schemes

NTF models

Definition
What do we mean by a tensor?

Tensor is an L-way array or simply a dataset indexed by L indices (L = 2

for matrices, L = 3 for boxes)

2-way array (matrix)

3-way array (tensor)

NOTE: In Physics tensors have a dierent meaning.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

109 / 170

Multi-stream and cross-modal NMF schemes

NTF models

Reminder on NMF models

vfn

wfk hkn ,

V = WH
H

=
W

h1
=

w1

Essid & Ozerov (TPT/Technicolor)

h2
+

w2

A tutorial on NMF

h3
+

w3

ICME 2014

110 / 170

Multi-stream and cross-modal NMF schemes

NTF models

CANDECOMP / PARAFAC (NTF) models

(Bro, 1997)

vfnl

wfk hkn qlk ,

hk

h1
w1

Essid & Ozerov (TPT/Technicolor)

qk

q1

wk

=
W
q2
+

q3

h2
w2

A tutorial on NMF

h3
w3

ICME 2014

111 / 170

Multi-stream and cross-modal NMF schemes

NTF models

TUCKER3
(Kiers, 2000)

vfnl

p,k,r

wfp hkn qrl gpkr

V

G is called a core tensor

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

112 / 170

Multi-stream and cross-modal NMF schemes

NTF models

Factor graphs representation for NTF I

(Yilmaz and Cemgil, 2010)

proposed representing various NTF models using

factor graphs (Loeliger, 2004), which are connected graphs:
Yilmaz and Cemgil 2010

with cycles and squares as nodes, and

where a vertex cannot connect two boxes or two cycles together (a

bipartite graph).

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

113 / 170

Multi-stream and cross-modal NMF schemes

NTF models

Factor graphs representation for NTF II

(Yilmaz and Cemgil, 2010)

Representing NTF models with factor graphs (Yilmaz and Cemgil, 2010):
Latent factors (e.g., W or H) are represented by square nodes.
Latent factor indices are represented by cycle nodes (in gray for
observed indices and in white for latent indices).
A vertex connecting a cycle node with a square node means that the
corresponding latent factor is indexed by the corresponding index.

vfn

NMF

wfk hkn

k
W

Essid & Ozerov (TPT/Technicolor)

P
H

A tutorial on NMF

ICME 2014

114 / 170

Multi-stream and cross-modal NMF schemes

NTF models

Factor graphs representation for NTF III

(Yilmaz and Cemgil, 2010)

vfn

NMF:

PARAFAC: vfnl

TUCKER3: vfnl
NMF
f

Pk

Pk

wfk hkn ,
wfk hkn qlk ,

p,k,r

wfp hkn qrl gpkr .

f
W

TUCKER3

PARAFAC
l

Essid & Ozerov (TPT/Technicolor)

l
p

r
k

H
n

f
Q

G
H

n
A tutorial on NMF

ICME 2014

115 / 170

Multi-stream and cross-modal NMF schemes

NTF models

Generalized NTF
(Yilmaz and Cemgil, 2010)

proposed to call Generalized NTF any nonnegative

factorization model that can be represented this way as a factor graph.
Yilmaz and Cemgil 2010

Advantages of such a general and graphical representation:

Having a graphical representation is always nice!

This representation generalizes many existing models (Yilmaz and Cemgil,

2010; Cemgil et al., 2011).

Results from the factor graph theory (e.g., the sum-product algorithm
Kschischang et al. 2001) can be re-used.

Given a graph, a corresponding optimization algorithm (e.g., MU rules)

can be generated automatically.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

116 / 170

Multi-stream and cross-modal NMF schemes

NTF models

Example of MU rules for TUCKER3 NTF

vfnl vfnl =

p,k,r

wfp hkn qrl gpkr

MU rules for TUCKER3 NTF with the

-divergence (one iteration):
hkn

wfp

qrl

gpkr

hkn

wfp

qrl

vfnl wfp qrl gpkr

f ,l ,p,r

fnl
n,l ,k,r vfnl v

fnl hkn qrl gpkr

n,l ,k,r v

vfnl wfp hkn gpkr

f
W

l
p
k

r
G

vfnl vfnl wfp hkn gpkr

f ,n,p,k

hkn qrl gpkr

f ,n,p,k

gpkr

vfnl vfnl wfp qrl gpkr

f ,l ,p,r

f ,n,l

vfnl vfnl wfp hkn qrl

f ,n,l

Essid & Ozerov (TPT/Technicolor)

vfnl wfp hkn qrl

A tutorial on NMF

ICME 2014

117 / 170

Multi-stream and cross-modal NMF schemes

NTF models

Surprising facts about PARAFAC

(Yilmaz and Cemgil, 2010)

There are a lot of interesting mathematical developments around NTF

(see, e.g., Lim and Comon 2010).
Surprisingly, but some results for the PARAFAC NTF (L-way arrays with
L 3) are quantitatively dierent from the results for the NMF.
Some surprising results on PARAFAC NTF:

In contrast to the NMF, the uniqueness conditions for the PARAFAC

NTF are mild (Kruskal, 1977).

In contrast to the NMF, there exist tensors for which rank-R

approximations (i.e., PARAFACs with R latent components) do not
exist (Lim and Comon, 2010). In other words, the set of tensors with rank
R is not guaranteed to be closed.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

118 / 170

Multi-stream and cross-modal NMF schemes

NTF models

Applications of NTF to audio-visual content processing

Audio

Music genre classification (Benetos and Kotropoulos, 2008)

Source separation (FitzGerald et al., 2008; Ozerov et al., 2011a)

Compression (Nikunen et al., 2011; Ozerov et al., 2013)

Videos and images

Action recognition (Kim and Cipolla, 2009; Krausz and Bauckhage, 2010)

Faces analysis (Shashua and Hazan, 2005)

Image retrieval (Liu et al., 2014)

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

119 / 170

Multi-stream and cross-modal NMF schemes

Co-factorisation schemes

I Introduction
I NMF models
I Algorithms for solving NMF
I Constrained NMF schemes
I Multi-stream and cross-modal NMF schemes
NTF models
Co-factorisation schemes
I Applications
I Conclusion
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

120 / 170

Multi-stream and cross-modal NMF schemes

Co-factorisation schemes

Motivation
Multimodal speaker diarization on edited videos

Observation: the audio and visual streams are related:

! Exploit both audio and visual features to perform speaker diarization.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

121 / 170

Multi-stream and cross-modal NMF schemes

Co-factorisation schemes

NMF for multimodal data analysis I

Possible approaches

NTF cannot be used: features from each modality do not live in spaces
of same dimensionality;

! an observation tensor cannot be built for such data.

Alternatively, concatenate the feature observations:

V1
V=
WH
V2

! same cost functions need to be used for dierent modalities: not always
optimal.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

122 / 170

Multi-stream and cross-modal NMF schemes

Co-factorisation schemes

NMF for multimodal data analysis II

Possible approaches

Another idea: perform co-factorisation constraining the activations to

be the same:
(
V1 W1 H
solving:
min D1 (V1 , W1 H) + 2 D2 (V2 , W2 H)
W1 ,W2 ,H
V2 W2 H

! does not account for possible local discrepancies across modalities:

! constrain the audio and visual data factorisations to be related:

temporal activations relating to these two streams of data should be
close, not necessarily equal.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

123 / 170

Multi-stream and cross-modal NMF schemes

Co-factorisation schemes

Soft nonnegative matrix co-factorisation

(Seichepine et al., 2013)

Solve the problem:

min

W1 ,H1 ,W2 ,H2

D1 (V1 | W1 H1 ) +

2 D2 (V2 | W2 H2 )+ c

kH1

H2 kp

D1 is a measure of fit penalizing the reconstruction errors for the first

modality ;

D2 is a measure of fit penalizing the reconstruction errors for the second

modality ;

kkp is a penalization term coupling factorizations for the first and the second
modality;

and

are weighting hyperparameters.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

124 / 170

Multi-stream and cross-modal NMF schemes

Co-factorisation schemes

Soft nonnegative matrix co-factorisation

(Seichepine et al., 2013)

Solve the problem:

min

W1 ,H1 ,W2 ,H2

D1 (V1 | W1 H1 ) +

2 D2 (V2 | W2 H2 )+ c

kH1

H2 kp

I Remarks:

similarly a dependency between W1 and W2 could be accounted for:

min D(V|WH) is equivalent to min D(VT |HT WT ).

if H1 and H2 have dierent dimensions, the penalty term can readily

ignore rows and columns of H1 that have no match in H2 .

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

125 / 170

Multi-stream and cross-modal NMF schemes

Co-factorisation schemes

Solving the problem

We have devised a block-coordinate MM algorithm solving the

problem:
H1 , H2 , W1 and W2 are updated sequentially;
the cost function is decreased at each iteration;

update rules have been determined for:

Kullback-Liebler and Itakura-Saito cost functions;
`2 and `1 -coupling penalties;
`2 and `1 -temporal smoothing penalties.

see (Seichepine et al., 2014b) for more details.

Matlab scripts are available on N. Seichepines web page:

http://www.telecom-paristech.fr/~seichepi

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

126 / 170

Multi-stream and cross-modal NMF schemes

Co-factorisation schemes

Applications

The method has been successfully applied (Seichepine et al., 2014b) to:

multimodal speaker diarization;

multi-channel musical audio source separation.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

127 / 170

Applications

I Introduction
I NMF models
I Algorithms for solving NMF
I Constrained NMF schemes
I Multi-stream and cross-modal NMF schemes
I Applications
Video structuring
Audio source separation
I Conclusion
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

128 / 170

Applications

Video structuring

The video structuring problem

Goal: automatically extract a temporal organization of a
document into units conveying a homogeneous type of
(audio/video) content.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

129 / 170

Applications

Video structuring

A generic video structuring system using NMF

Challenge: perform the task in a non-supervised fashion.
Proposed approach: a generic structuring scheme using NMF (Essid and
Fevotte, 2013):
images/
audio segments

A/V
frames

Histograms
of words

Word vocab.
extraction

Vocab.

NMF

Activation
thresholding

Structure extracted

1. Bag of words representation

1. create a low-level (visual/audio) vocabulary and use it to extract

histogram of (visual/audio) words from the sequence of observation
frames;
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

130 / 170

Applications

Video structuring

A generic video structuring system using NMF

Challenge: perform the task in a non-supervised fashion.
Proposed approach: a generic structuring scheme using NMF (Essid and
Fevotte, 2013):

A/V
frames

Histograms
of words

Word vocab.
extraction

Vocab.

KL-NMF

Activation
thresholding

Structure extracted

2. Data factorisation

2. apply a variant of smooth NMF using the Kullback-Leibler divergence

to extract latent structuring events and their activations across the
duration of the document.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

130 / 170

Applications

Video structuring

A generic video structuring system using NMF

Challenge: perform the task in a non-supervised fashion.
Proposed approach: a generic structuring scheme using NMF (Essid and
Fevotte, 2013):

A/V
frames

Histograms
of words

Word vocab.
extraction

Vocab.

KL-NMF

Activation
thresholding

Structure extracted

2. Data factorisation

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

130 / 170

Applications

Video structuring

A generic video structuring system using NMF

Challenge: perform the task in a non-supervised fashion.
Proposed approach: a generic structuring scheme using NMF (Essid and
Fevotte, 2013):

A/V
frames

Histograms
of words

Word vocab.
extraction

Vocab.

KL-NMF

Activation
thresholding

Structure extracted

2. Data factorisation

Activations should be temporally smooth: structuring events naturally exhibit a

certain temporal continuity.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

130 / 170

Applications

Video structuring

Applications
Onscreen person-oriented structuring

Discover the video editing structure: label the video frames as follows in a
non-supervised fashion:
"Full group"

"Participant 1"

"Participant 3"

"Multiple participants" "Multiple participants"

"Participant 2"

"Participant 2"

"Participant 4"

"Participant 5"

Using the Canal9 political debates database (Vinciarelli et al., 2009).

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

131 / 170

Applications

Video structuring

Applications
Speaker diarization

Who spoke when?

A notable diculty: handling overlapped speech segments.

! NMF has the potential to alleviate this issue.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

132 / 170

Applications

Video structuring

Experimental validation

Canal9 political debates database (Vinciarelli et al., 2009)

broadcasts featuring a moderator and 2 to 4 guests;
moderators, guest and background vary;
7 hours of video content: 10 minutes from each of the first 41 shows;
189 distinct persons; 28521 video shots.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

133 / 170

Applications

Video structuring

Visual features

Visual vocabulary creation

PHOW features (Bosch et al., 2007): histograms of
orientation gradients over 3 scales, on 8-pixel step
grid; extracted from faces and clothing regions,
determined automatically for current video;
quantization over 128 bins using K-means.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

134 / 170

Applications

Video structuring

Evaluation
Reference system: ergodic Hidden Markov Models (HMM) states:
Nsp + 2 states, Nsp : number of speakers;
Gaussian-emission probabilities with full covariance matrices;
same features as NMF system.

NMF Parameters:
K = Nsp + 1;
best (in terms of cost-function value) of 10 random initializations;
3 values of smoothing penalty s are tested: s 2 {0; 0.1; 1}.

Scoring:

using frame-type classification error rate;

computed following the NIST speaker diarization error procedure (performs a
one-to-one mapping between groundtruth and automatically determined
segment labels).
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

135 / 170

Applications

Video structuring

Results
Shot-type classification error rates

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

136 / 170

Applications

Video structuring

Results
Visualising the activations
Full group

MP

Speaker 5 Full group

MP

MP:
Multiple Participants

Full group
Speaker 1

Speaker 2

Speaker 3

Speaker 4
Speaker 5

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

137 / 170

Applications

Video structuring

NMF for speaker diarization

(Essid and Fevotte, 2013)

...
State histograms
Data frames
States decoded s1 s2 s3
Analysis windows

...

...

... st1 st

Audio waveform
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

138 / 170

Applications

Video structuring

Evaluation
(Seichepine et al., 2014b)

Dataset and scoring:

using the Canal9 political debates videos;
using the NIST DER (Diarization Error Rate).

Reference methods:
a simple K-means applied to data matrices V;
a state-of-the-art GMM-based diarization system: the LIUM Speaker
Diarization system (Meignier and Merlin, 2010).

NMF Parameters:
audio: Nsp components; video: Nsp + 1 components;
initializations based on output of previously computed monomodal NMFs;
1 = 0.02, 2 = 0.2 and c = 0.1, respectively for visual, audio and coupling
penalties; tuned on development data.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

139 / 170

Applications

Video structuring

Results

Method

K-means

NMF

S-NMF

CS-NMF

Mean score
Mean (3 speakers)
Mean (4 speakers)
Mean (5 speakers)
Prop. better than K-means

13.51
15.86
10.90
12.46
/

14.13
16.37
13.65
12.67
30%

11.45
17.40
10.69
7.43
85%

10.24
15.92
9.29
6.46
85%

S-NMF: `1 -smoothed NMF;

CS-NMF: `1 -coupled `1 -smoothed NMF.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

140 / 170

Applications

Video structuring

Results

Method

K-means

NMF

S-NMF

CS-NMF

LIUM

Mean score
Mean (3 speakers)
Mean (4 speakers)
Mean (5 speakers)
Prop. better than K-means
Prop. better than LIUM

13.51
15.86
10.90
12.46
/
0%

14.13
16.37
13.65
12.67
30%
0%

11.45
17.40
10.69
7.43
85%
30%

10.24
15.92
9.29
6.46
85%
52%

6.87
7.67
8.26
5.97
100%
/

S-NMF: `1 -smoothed NMF;

CS-NMF: `1 -coupled `1 -smoothed NMF.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

141 / 170

Applications

Audio source separation

I Introduction
I NMF models
I Algorithms for solving NMF
I Constrained NMF schemes
I Multi-stream and cross-modal NMF schemes
I Applications
Video structuring
Audio source separation
I Conclusion
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

142 / 170

Applications

Audio source separation

NMF for audio source separation

Main idea

Spectral
patterns

frequency

Mixture
spectrogram

Temporal
activations
=

H = [H1 ; H2 ]
time
V

W = [W1 , W2 ]

Source 1 model
H1

W1

Essid & Ozerov (TPT/Technicolor)

Source 2 model
H2

W2

A tutorial on NMF

ICME 2014

143 / 170

Applications

Audio source separation

NMF for audio source separation

Details

All audio signals are represented in the complex-values short time Fourier
transform (STFT) domain (a time-frequency representation).
Problem: Given a mixture of two sources
X = S1 + S2 ,
estimate S1 and S2 .

X, S1 , S2 2 CF N ,

Basic approach:

.2

Compute an NMF decomposition V = |X| WH = W1 H1 + W2 H2 .

Compute source estimates by Wiener filtering:
W1 H1
W2 H2
b1 =
b2 =
S
X,
S
W1 H1 + W2 H2
W1 H1 + W2 H2

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

X.

ICME 2014

144 / 170

Applications

Audio source separation

NMF for audio source separation

Details

Main diculty: How to compute the decomposition

V WH = W1 H1 + W2 H2
such that (W1 , H1 ) and (W2 , H2 ) represent well the sources S1 and S2 ,
respectively?
One popular approach:

Compute W1 and W2 from some training samples (e.g., downloaded

from the internet).

Set W = [W1 , W2 ] and fix it during mixture decomposition.

Problem: Training samples are not always available and/or representative.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

145 / 170

Applications

Audio source separation

Source separation demo

On-the-fly audio source separation (El Badawy et al., 2014)

A user queries audio samples from the internet to pre-train W1 and W2 .

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

146 / 170

Applications

Audio source separation

User-guided audio source separation

(Ozerov et al., 2011a; Duong et al., 2014)

A user is simply asked to annotate temporal segments of source

activities (active or non-active).
H

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

147 / 170

Applications

Audio source separation

User-guided audio source separation

Demo (Ozerov et al., 2011a; Duong et al., 2014)

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

148 / 170

Applications

Audio source separation

Text-informed audio source separation

General scheme (Le Magoarou et al., 2013)

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

149 / 170

Applications

Audio source separation

Text-informed audio source separation

Coupled NMF-based approach (Le Magoarou et al., 2013)

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

150 / 170

Applications

Audio source separation

Text-informed audio source separation

Demo (Le Magoarou et al., 2013)

Mixture (speech + background)

Estimated speech

Synthesized speech example

Estimated background

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

151 / 170

Conclusion

Summary and future challenges

Take-home messages I

NMF is a versatile data decomposition technique that has proven

eective for diverse applications across numerous disciplines,
it tends to provide meaningful and natural part-based data
representations,
it can be used both for feature learning, topic extraction, clustering,
segmentation, source separation, coding...

For NMF to be successful, it has to be estimated using appropriate

cost-functions reflecting prior knowledge about the data.

Being non-unique, NMF should incorporate constraints relating to the

data, either though:
regularized cost-functions accounting for sparsity, shape, smoothness,
cross-modal dependency constraints..., or
alternative formulations, e.g, geometric approaches having the potential to
estimate exact NMF models.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

152 / 170

Conclusion

Summary and future challenges

Take-home messages II

Many algorithms are available to estimate NMF, mostly alternating

updates of W and H; variants include:
multiplicative updates: heuristic, simple and easy to implement, but slow
and instable,
majorisation-minimisation: well-founded for a variety of cost functions,
stable, still slow,
gradient-descent and Newton: fast but unstable.

NMF is a state-of-the-art technique for a number of audio-processing

tasks (transcription, source separation...),

it has a great potential for video (and RBG+depth) analysis tasks,

especially temporal structure analysis.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

153 / 170

Conclusion

Summary and future challenges

Ongoing and future research

How to properly estimate the model-order K ?

How to achieve better and faster convergence?

How to perform non-linear data decompositions?

How to handle big data?

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

154 / 170

Conclusion

NMF software & bibliography

A selection of NMF software

Software

Language

beta_ntf

Python

sklearn.decomposition.NMF

Python

`2 -norm, gradient-descent, sparsity

IMM DTU NMF toolbox

Matlab

`2 -norm, MM, gradient-descent, ALS

Fvottes matlab scripts

Matlab

`2 -norm, KL and IS-div, MM, probabilistic

Seichepines matlab
scripts

Matlab

svmnmf

Matlab

libNMF

Essid & Ozerov (TPT/Technicolor)

Main features
Weighted tensor decomposition, all
-divergences, MM

Soft co-factorisation, `2 -norm, KL and IS-div,

`1 /`2 -norm temporal smoothing, MM
Geometric SVM-based NMF, kernel-based
non-linear decompositions, fast
`2 -norm, MM, gradient-descent, ALS,
multi-core, fast

A tutorial on NMF

ICME 2014

155 / 170

Conclusion

NMF software & bibliography

Bibliography I
A. Cichocki, S. Cruces, and S. Amari. Generalized {A}lpha-{B}eta Divergences and Their Application to Robust
Nonnegative Matrix Factorization. Entropy, 13:134170, 2011.
A. Lefvre, F. Bach, and C. Fvotte. {I}takura-{S}aito nonnegative matrix factorization with group sparsity. In
Proc. International Conference on Acoustics, Speech and Signal Processing (ICASSP), Prague, Czech Republic,
2011. URL http://perso.telecom-paristech.fr/~fevotte/Proceedings/icassp11c.pdf.
S. A. Abdallah and M. D. Plumbley. Polyphonic transcription by nonnegative sparse coding of power spectra. In
Proc. 5th International Symposium Music Information Retrieval (ISMIR04), pages 318325, Barcelona, Spain,
2004.
R. Albright, J. Cox, D. Duling, A. Langville, and C. Meyer. Algorithms, initializations, and convergence for the
nonnegative matrix factorization. Technical Report Math 81706, NCSU, 2006.
S. Arberet, A. Ozerov, F. Bimbot, and R. Gribonval. A tractable framework for estimating and combining spectral
source models for audio source separation. Signal Processing, 92(8):18861901, 2012.
M. Arngren, M. Schmidt, and J. Larsen. Unmixing of hyperspectral images using Bayesian non-negative matrix
factorization with volume prior. Journal of Signal Processing Systems, 65(3):479496, 2011.
R. Badeau, N. Bertin, and E. Vincent. Stability analysis of multiplicative update algorithms and application to
non-negative matrix factorization. IEEE Transactions on Neural Networks, 21(12):18691881, Dec. 2010.
E. Benetos and C. Kotropoulos. A tensor-based approach for automatic music genre classification. In Proceedings of
the European Signal Processing Conference, Lausanne, Switzerland, 2008.
M. W. Berry and M. Browne. Email Surveillance Using Non-negative Matrix Factorization. Computational and
Mathematical Organization Theory, 11(3):249264, Jan. 2006. ISSN 1381-298X. doi:
10.1007/s10588-005-5380-5. URL http://link.springer.com/10.1007/s10588-005-5380-5.
M. W. Berry, M. Browne, A. N. Langville, V. P. Pauca, and R. J. Plemmons.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

156 / 170

Conclusion

NMF software & bibliography

Bibliography II
N. Bertin, R. Badeau, and E. Vincent. Enforcing Harmonicity and Smoothness in Bayesian Non-Negative Matrix
Factorization Applied to Polyphonic Music Transcription. IEEE Transactions on Audio, Speech, and Language
Processing, 18(3):538549, Mar. 2010. ISSN 1558-7916. doi: 10.1109/TASL.2010.2041381. URL
http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5410052.
V. D. Blondel, N.-D. Ho, and P. V. Dooren. Weighted non-negative matrix factorization and face feature extraction.
In Image and Vision Computing, 2008.
A. Bosch, A. Zisserman, and X. Munoz. Image classification using random forests and ferns. In IEEE 11th
International Conference on Computer Vision. IEEE, 2007. URL
http://www.computer.org/portal/web/csdl/doi/10.1109/ICCV.2007.4409066.
C. Boutsidis and E. Gallopoulos. SVD based initialization: A head start for nonnegative matrix factorization.
Pattern Recognition, 41:13501362, 2008.
L. M. Bregman. The relaxation method of finding the common points of convex sets and its application to the
solution of problems in convex programming. USSR Comput. Math. Math. Phys., 7(3):210217, 1967.
R. Bro. PARAFAC. Tutorial and applications. Chemometrics and Intelligent Laboratory Systems, 38:149171, Oct.
1997.
J.-P. Brunet, P. Tamayo, T. R. Golub, and J. P. Mesirov. Metagenes and Molecular Pattern Discovery Using Matrix
Factorization. In Proceedings of the National Academy of Sciences, pages 41644169, 2004.
S. Bucak and B. Gunsel. Incremental subspace learning via non-negative matrix factorization. Pattern Recognition,
42(5):788797, May 2009.
S. S. Bucak and B. Gunsel. Video Content Representation by Incremental Non-Negative Matrix Factorization. In
2007 IEEE International Conference on Image Processing, volume 2, pages II 113II 116. IEEE, 2007. ISBN
978-1-4244-1436-9. doi: 10.1109/ICIP.2007.4379105. URL
http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4379105.
D. Cai, X. He, J. Han, and T. Huang. Graph regularized nonnegative matrix factorization for data representation.
IEEE Trans. on Pattern Analysis and Machine Intelligence, 33:15481560, 2011.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

157 / 170

Conclusion

NMF software & bibliography

Bibliography III
A. T. Cemgil. Bayesian Inference for Nonnegative Matrix Factorisation Models, Feb. 2009a. URL
http://www.hindawi.com/journals/cin/2009/785152.abs.html.
A. T. Cemgil. Bayesian Inference for Nonnegative Matrix Factorisation Models. Computational Intelligence and
Neuroscience, 2009(Article ID 785152):17 pages, 2009b.
A. T. Cemgil, U. Simsekli, and Y. C. Subakan. Probabilistic latent tensor factorization framework for audio modeling.
In Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA11), 2011.
J.-C. Chen. The nonnegative rank factorizations of nonnegative matrices. 62:207217, Nov 1984. ISSN 00243795.
doi: 10.1016/0024-3795(84)90096-X. URL
http://www.sciencedirect.com/science/article/pii/002437958490096X.
S. Choi. Algorithms for orthogonal nonnegative matrix factorization. In Proceedings of the International Joint
Conference on Neural Networks (IJCNN), 2008.
a. Cichocki and T. Rutkowski. Constrained non-Negative Matrix Factorization Method for EEG Analysis in Early
Detection of Alzheimer Disease. 2006 IEEE International Conference on Acoustics Speed and Signal Processing
Proceedings, 5(4):V893V896, 2006. doi: 10.1109/ICASSP.2006.1661420. URL
http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1661420.
A. Cichocki, R. Zdunek, and S. Amari. Csiszars divergences for non-negative matrix factorization: Family of new
algorithms. In Proc. 6th International Conference on Independent Component Analysis and Blind Signal
Separation (ICA06), pages 3239, Charleston SC, USA, 2006.
A. Cichocki, Y.-D. K. H. Lee, and S. Choi. Non-negative matrix factorization with -divergence. Pattern Recognit.
Lett., 29:14331440, 2008.
O. Cirakman, B. Gunsel, N. Sengor, and O. Gursoy. Key-frame based video fingerprinting by NMF. In 2010 IEEE
International Conference on Image Processing, pages 23732376. IEEE, Sept. 2010. ISBN 978-1-4244-7992-4.
doi: 10.1109/ICIP.2010.5652649. URL
http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5652649.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

158 / 170

Conclusion

NMF software & bibliography

Bibliography IV
M. Cooper and J. Foote. Summarizing Video using Non-Negative Similarity Matrix Factorization. In Proc. IEEE
Workshop on Multimedia Signal Processing, volume 00, pages 25, 2002. ISBN 0780377141.
C. Damon, A. Liutkus, A. Gramfort, and S. Essid. Nonnegative Matrix Factorization for Single-Channel EEG
Artifact Rejection. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP),
Vancouver, Canada, 2013a.
C. Damon, A. Liutkus, A. Gramfort, and S. Essid. Nonnegative Tensor Factorization for Single-Channel EEG
Artifact Rejection. In IEEE International Workshop on Machine Learning for Signal Processing, Southampton,
UK, 2013b.
T. V. de Cruys. A non-negative tensor factorization model for selectional preference induction. Natural Language
Engineering, 16(4):417437, 2010.
K. Devarajan and N. Ebrahimi. Molecular pattern discovery using non-negative matrix factorization based on Renyis
information measure. In Proceedings of the XII SCMA International Conference, Auburn, Alabama, December
2005.
C. Ding, X. He, and H. Simon. On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering. In
SIAM Data Mining Conference, number 4, 2005. URL
http://epubs.siam.org/doi/abs/10.1137/1.9781611972757.70.
C. Ding, W. P. T. Li, and H. Park. Orthogonal nonnegative matrix tri-factorizations for clustering. In Proc. of the
12th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, pages 126135, 2006.
C. H. Ding, T. Li, and M. I. Jordan. Convex and Semi-Nonnegative Matrix Factorizations. Pattern Analysis and
Machine Intelligence, IEEE Transactions on, 32(1):4555, 2010. ISSN 0162-8828. doi:
10.1109/TPAMI.2008.277. URL
http://ieeexplore.ieee.org/xpls/abs_all.jsp?isnumber=5339303&arnumber=4685898&count=16&index=4.
K. Drakakis, S. Rickard, R. de Frein, and A. Cichocki. Analysis of Financial Data using Non-Negative Matrix
Factorization. International Journal of Mathematical Sciences, 6(2), 2007.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

159 / 170

Conclusion

NMF software & bibliography

Bibliography V
N. Q. K. Duong, A. Ozerov, and L. Chevallier. Temporal annotation-based audio source separation using weighted
nonnegative matrix factorization. In 4th IEEE International Conference on Consumer Electronics - Berlin (IEEE
2014 ICCE-Berlin), 2014.
J. Durrieu, A. Ozerov, and C. Fvotte. Main instrument separation from stereophonic audio signals using a
source/filter model. European Signal Processing Conference (EUSIPCO), 2009. URL
http://www.quaero.org/media/files/bibliographie/eusipco09_2.pdf.
E. Vincent, N. Bertin, and R. Badeau. Harmonic and inharmonic Nonnegative Matrix Factorization for Polyphonic
Pitch transcription. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing
(ICASSP08), pages 109112, 2008. doi: 10.1109/ICASSP.2008.4517558.
J. Eggert and E. Korner. Sparse coding and NMF. In 2004 IEEE International Joint Conference on Neural Networks
(IEEE Cat. No.04CH37541), volume 4, pages 25292533. IEEE, 2004. ISBN 0-7803-8359-1. doi:
10.1109/IJCNN.2004.1381036. URL
http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1381036.
S. Eguchi and Y. Kano. Robustifying maximum likelihood estimation. Technical report, Institute of Statistical
Mathematics, June 2001. Research Memo. 802.
D. El Badawy, N. Q. K. Duong, and A. Ozerov. On-the-fly audio source separation. In Proc. IEEE Int. Workshop on
Machine Learning for Signal Processing (MLSP), Reims, France, Sept. 2014.
S. Essid. A single-class SVM based algorithm for computing an identifiable NMF. In IEEE International Conference
on Acoustics, Speech and Signal Processing, Kyoto, Japan, 2012.
S. Essid and C. Fevotte. Decomposing the Video Editing Structure of a Talk-show using Nonnegative Matrix
Factorization. In International Conference on Image Processing (ICIP), Orlando, FL, USA, 2012.
S. Essid and C. Fevotte. Smooth Nonnegative Matrix Factorization for Unsupervised Audiovisual Document
Structuring. IEEE Transactions on Multimedia, 15(2):415425, 2013. ISSN 1520-9210. doi:
10.1109/TMM.2012.2228474.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

160 / 170

Conclusion

NMF software & bibliography

Bibliography VI
C. Fevotte. Slides of lecture on unsupervised data decompositions. Technical report, CNRS/LTCI, Telecom
ParisTech, 2012.
C. Fvotte and J. Idier. Algorithms for nonnegative matrix factorization with the beta-divergence. Oct. 2010. URL
http://arxiv.org/abs/1010.1763.
C. Fvotte and A. Ozerov. Notes on nonnegative tensor factorization of the spectrogram for audio source
separation: statistical insights and towards self-clustering of the spatial cues. In Proc. 7th International
Symposium on Computer Music Modeling and Retrieval (CMMR), volume 5493 of Lecture Notes in Computer
Science, pages 102115, Malaga, Spain, 2010. Springer. URL
http://perso.telecom-paristech.fr/~fevotte/Proceedings/cmmr10.pdf.
C. Fevotte, N. Bertin, and J.-L. Durrieu. Nonnegative Matrix Factorization with the Itakura-Saito Divergence. With
Application to Music Analysis. Neural Computation, 21(3), Mar. 2009.
C. Fvotte, N. Bertin, and J.-L. Durrieu. Nonnegative matrix factorization with the {I}takura-{S}aito divergence.
{W}ith application to music analysis. Neural Computation, 21(3):793830, 2009. doi:
10.1162/neco.2008.04-08-771. URL http://www.tsi.enst.fr/~fevotte/Journals/neco09_is-nmf.pdf.
D. FitzGerald, M. Cranitch, and E. Coyle. Extended Nonnegative Tensor Factorisation Models for Musical Sound
Source Separation. Computational Intelligence and Neuroscience, 2008(Article ID 872425):15 pages, 2008. doi:
10.1155/2008/872425.
H. Fujisawa and S. Eguchi. Robust parameter estimation with a small bias against heavy contamination. Journal of
Multivariate Analysis, 99(9):20532081, Oct. 2008.
Y. Gao and G. Church. Improving molecular cancer class discovery through sparse non-negative matrix factorization.
Bioinformatics, 21:39703975, 2005. doi: doi:10.1093/bioinformatics/bti653.
E. Gaussier and C. Goutte. Relation between PLSA and NMF and implications. In Proc. 28th annual international
ACM SIGIR conference on Research and development in information retrieval (SIGIR05), pages 601602, New
York, NY, USA, 2005. ACM. ISBN 1595930345. URL http://dl.acm.org/citation.cfm?id=1076148.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

161 / 170

Conclusion

NMF software & bibliography

Bibliography VII
N. Gillis. Regularization, Optimization, Kernels, and Support Vector Machines, chapter The why and how of
nonnegative matrix factorization. Chapman & Hall/CRC, 2014.
D. Greene, G. Cagney, N. Krogan, and P. Cunningham. Ensemble Non-negative Matrix Factorization Methods for
Clustering Protein-Protein Interactions. Bioinformatics, 24(15):17221728, 2008.
R. Hennequin, B. David, and R. Badeau. Score informed audio source separation using a parametric model of
non-negative spectrogram. In International Conference on Acoustics, Speech, and Signal Processing, 2011. URL
http://hal.inria.fr/docs/00/94/52/94/PDF/hennequin_icassp2011.pdf.
T. Hofmann. Probabilistic latent semantic analysis. Proceedings of the Fifteenth conference on Uncertainty . . . ,
1999. URL http://dl.acm.org/citation.cfm?id=2073829.
P. O. Hoyer. Non-negative Matrix Factorization with Sparseness Constraints. The Journal of Machine Learning
Research, 5:14571469, Dec. 2004. ISSN 1532-4435. URL http://dl.acm.org/citation.cfm?id=1005332.1044709.
D. R. Hunter and K. Lange. A tutorial on mm algorithms. Amer. Stat., 58(1):3037, Feb. 2004.
I. S. Dhillon and S. Sra. Generalized Nonnegative Matrix Approximations with {B}regman Divergences. Advances in
Neural Information Processing Systems (NIPS), 19, 2005.
M. Jeter and W. Pye. A note on nonnegative rank factorizations. Linear Algebra and its Applications, 38:171173,
Jun 1981. ISSN 00243795. doi: 10.1016/0024-3795(81)90018-5. URL
http://www.sciencedirect.com/science/article/pii/0024379581900185.
S. Jia and Y. Qian. Constrained Nonnegative Matrix Factorization for Hyperspectral Unmixing. IEEE Transactions
on Geoscience and Remote Sensing, 47(1):161173, Jan. 2009. ISSN 0196-2892. doi:
10.1109/TGRS.2008.2002882. URL
http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4694061.
M. M. Kalayeh, H. Idrees, and M. Shah. NMF-KNN : Image Annotation using Weighted Multi-view Non-negative
Matrix Factorization. In CVPR, 2014.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

162 / 170

Conclusion

NMF software & bibliography

Bibliography VIII
A. Karatzoglou, X. Amatriain, L. Baltrunas, and N. O. . Multiverse recommendation: N-dimensional tensor
factorization for context-aware collaborative filtering. In ACM, editor, Proceedings of the 4th ACM Conference
on Recommender Systems (RecSys10), pages 7986, New York, NY, 2010.
H. A. L. Kiers. Towards a standardized notation and terminology in multiway analysis. Journal of Chemometrics, 14:
105122, 2000.
J. Kim and H. Park. Sparse Nonnegative Matrix Factorization for Clustering. Technical report, Georgia Institute of
Technology, 2008. URL https://smartech.gatech.edu/handle/1853/20058.
T.-K. Kim and R. Cipolla. Canonical correlation analysis of video volume tensors for action categorization and
detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(8):14151428, 2009.
Y.-D. Kim and S. Choi. A Method of Initialization for Nonnegative Matrix Factorization. In IEEE International
Conference on Acoustics, Speech and Signal Processing (ICASSP07), volume 2, pages 537540, Honolulu,
Hawaii, 2007.
B. Klingenberg, J. Curry, and A. Dougherty. Non-negative matrix factorization: Ill-posedness and a geometric
algorithm. Pattern Recognition, 42(5):918928, May 2009. ISSN 0031-3203. doi:
10.1016/j.patcog.2008.08.026. URL http://linkinghub.elsevier.com/retrieve/pii/S0031320308003403http:
//www.sciencedirect.com/science/article/B6V14-4TCR1KR-1/2/8bedc245dc0fd9ba7487561f8df431cahttp:
//www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V14-4TCR1KR-1&_user=771355&_coverDate=05/31/
2009&_rdoc=1&_fmt=high&_orig=search&_sort=d&_docanchor=&view=c&_searchStrId=1272450458&_rerunOrigin=
google&_acct=C000028498&_version=1&_urlVersion=0&_userid=771355&md5=35ba172c08afbfa2b0676a0f2dca1897.
B. Krausz and C. Bauckhage. Action Recognition in Videos Using Nonnegative Tensor Factorization. In 2010 20th
International Conference on Pattern Recognition, pages 17631766. IEEE, Aug. 2010. ISBN 978-1-4244-7542-1.
doi: 10.1109/ICPR.2010.435. URL http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5597190.
J. B. Kruskal. Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic
complexity and statistics. Linear Algebra and its Applications, 18(2):95 138, 1977.
F. Kschischang, B. Frey, and H.-A. Loeliger. Factor graphs and the sum-product algorithm. IEEE Transactions on
Information Theory, 47(2):498519, 2001.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

163 / 170

Conclusion

NMF software & bibliography

Bibliography IX
L. Le Magoarou, A. Ozerov, and N. Q. K. Duong. Text-informed audio source separation using nonnegative matrix
partial co-factorization. In Proc IEEE. Int. Workshop on Machine Learning for Signal Processing (MLSP), 2013.
D. D. Lee and H. S. Seung. Learning the parts of objects with nonnegative matrix factorization. Nature, 401:
788791, 1999.
D. D. Lee and H. S. Seung. Algorithms for non-negative matrix factorization. In Advances in Neural and
Information Processing Systems 13, pages 556562, 2001.
H. Lee, Y.-D. Kim, A. Cichocki, and S. Choi. Nonnegative tensor factorization for continuous EEG classification.
International Journal of Neural Systems, 17(4):305317, 2007.
H. Lee, A. Cichocki, and S. Choi. Kernel nonnegative matrix factorization for spectral EEG feature extraction.
Neurocomputing, 72(13-15):31823190, Aug. 2009. ISSN 09252312. doi: 10.1016/j.neucom.2009.03.005.
URL http://linkinghub.elsevier.com/retrieve/pii/S0925231209000757.
Y. Li, D. Sima, S. V. Cauter, A. C. Sava, U. Himmelreich, Y. Pi, and S. V. Huel. Hierarchical non-negative matrix
factorization (hNMF): A tissue pattern dierentiation method for glioblastoma multiforme diagnosis using MRSI.
NMR in Biomedicine, 26:307319, 2013.
L.-H. Lim and P. Comon. Multiarray signal processing: Tensor decomposition meets compressed sensing.
Compte-Rendus de lAcademie des Sciences, section Mecanique, 338(6):311320, June 2010.
A. Limem, G. Delmaire, M. Puigt, G. Roussel, and D. Courcot. Non-negative matrix factorization using weighted
Beta divergence and equality constraints for industrial source apportionment. In 23rd IEEE International
Workshop on Machine Learning for Signal Processing (MLSP 2013), Southampton, UK, September 2225 2013.
C.-J. Lin. On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization. IEEE
Transactions on Neural Networks, 18:15891596, 2007a.
C.-J. Lin. Projected Gradient Methods for Nonnegative Matrix Factorization. Neural Computation, 19:27562779,
2007b.
J. Liu, C. Wang, J. Gao, and J. Han. Multi-view clustering via joint nonnegative matrix factorization. In Proc. of
SDM, 2013. URL http://epubs.siam.org/doi/abs/10.1137/1.9781611972832.28.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

164 / 170

Conclusion

NMF software & bibliography

Bibliography X
X. Liu, Q. Xu, S. Yan, G. W. H. Jin, and S.-W. Lee. Nonnegative tensor co-factorization and its unified solution.
IEEE Transactions on Image Processing (TIP), 2014.
H. A. Loeliger. An introduction to factor graphs. IEEE Signal ProcessingMagazine, 21:2841, 2004.
W. Lu, W. Sun, and H. Lu. Robust watermarking based on dwt and nonnegative matrix factorization. Computers
and Electrical Engineering, 35(1):183188, 2009.
J. Mairal, F. Bach, J. Ponce, and G. Sapiro. Online learning for matrix factorization and sparse coding. The Journal
of Machine Learning Research, 11(10-60), 2010. URL http://dl.acm.org/citation.cfm?id=1756008.
A. Masurelle, S. Essid, and G. Richard. Gesture recognition using a NMF-based representation of motion-traces
extracted from depth silhouettes. In IEEE International Conference on Acoustics, Speech, and Signal Processing
(ICASSP), Florence, Italy, 2014.
S. Meignier and T. Merlin. {LIUM SPKDIARIZATION}: AN OPEN SOURCE TOOLKIT FOR DIARIZATION. In
CMU SPUD Workshop, Texas, USA, 2010. URL http://lium3.univ-lemans.fr/diarization/doku.php/welcome.
P. Melville and V. Sindhwai. Recommender systems. In C. Sammut and W. G., editors, Encyclopedia of Machine
Learning. Springer-Verlag, 2010. URL http://dl.acm.org/citation.cfm?id=245121.
L. Miao and H. Qi. Endmember extraction from highly mixed data using minimum volume constrained nonnegative
matrix factorization. IEEE Trans. on Geoscience and Remote Sensing, 47:765777, 2007.
N. Mohammadiha, P. Smaragdis, and A. Leijon. Supervised and unsupervised speech enhancement using NMF.
IEEE Transactions on Audio Speech and Language Processing, 21(10):21402151, Oct. 2013.
V. Monga and M. K. Mihcak. Robust and secure image hashing via non-negative matrix factorizations. IEEE Trans.
on information Forensics and Security, 2(3):376390, Sep. 2007.
J. Nikunen, T. Virtanen, and M. Vilermo. Multichannel audio upmixing based on non-negative tensor factorization
representation. In IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA11),
pages 3336, 2011.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

165 / 170

Conclusion

NMF software & bibliography

Bibliography XI
A. Ozerov and C. Fevotte. Multichannel nonnegative matrix factorization in convolutive mixtures for audio source
separation. IEEE transactions on Audio, Speech, and Language Processing, 18(3):550563, 2010. URL
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5229304.
A. Ozerov, C. Fvotte, and M. Charbit. Factorial scaled hidden Markov model for polyphonic audio representation
and source separation. In IEEE Workshop on Applications of Signal Processing to Audio and Acoustics
(WASPAA09), Mohonk, NY, Oct. 18-21 2009.
A. Ozerov, C. Fvotte, R. Blouet, and J.-L. Durrieu. Multichannel nonnegative tensor factorization with structured
constraints for user-guided audio source separation. In Proc. International Conference on Acoustics, Speech and
Signal Processing (ICASSP), Prague, Czech Republic, 2011a. URL
http://perso.telecom-paristech.fr/~fevotte/Proceedings/icassp11d.pdf.
A. Ozerov, A. Liutkus, R. Badeau, and G. Richard. Informed source separation: source coding meets source
separation. In IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA11),
Mohonk, NY, Oct. 2011b.
A. Ozerov, E. Vincent, and F. Bimbot. A general flexible framework for the handling of prior information in audio
source separation. IEEE Transactions on Audio, Speech and Language Processing, 20(4):11181133, 2012.
A. Ozerov, A. Liutkus, R. Badeau, and G. Richard. Coding-Based Informed Source Separation: Nonnegative Tensor
Factorization Approach. IEEE Transactions on Audio, Speech, and Language Processing, 21(8):16991712, Aug.
2013. ISSN 1558-7916. doi: 10.1109/TASL.2013.2260153. URL
http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6508860.
P. Paatero and U. Tapper. Positive matrix factorization: A non-negative factor model with optimal utilization of
error estimates of data values. Environmetrics, 5(2):111126, Jun 1994. ISSN 11804009. doi:
10.1002/env.3170050203. URL http://doi.wiley.com/10.1002/env.3170050203.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss,
V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn:
Machine learning in Python. Journal of Machine Learning Research, 12:28252830, 2011.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

166 / 170

Conclusion

NMF software & bibliography

Bibliography XII
Z. Rafii, D. Sun, F. Germain, and G. Mysore. Combining modeling of singing voice and background music for
automatic separation of musical mixtures. In 14th International Society for Music Information Retrieval (ISMIR),
Curitiba, PR, Brazil, 2013.
S. Rendle, L. B. Marinho, A. Nanopoulos, and . L.S. Thieme. Learning optimal ranking with tensor factorization for
tag recommendation. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining, pages 727736, 2009.
J. L. Roux, H. Kameoka, N. Ono, A. de Cheveign, and S. Sagayama. Computational auditory induction as a
missing-data model-fitting problem with Bregman divergence. Speech Communication, 53(5):658676,
May-June 2011.
R. Sandler and M. Lindenbaum. Nonnegative Matrix Factorization with Earth Movers Distance Metric for Image
Analysis. IEEE transactions on pattern analysis and machine intelligence, 33(8):15901602, Jan. 2011. ISSN
1939-3539. doi: 10.1109/TPAMI.2011.18. URL http://www.ncbi.nlm.nih.gov/pubmed/21263163.
M. N. Schmidt and M. Morup. Infinite non-negative matrix factorizations. In Proc. European Signal Processing
Conference (EUSIPCO), 2010.
M. N. Schmidt, J. Larsen, and F.-T. Hsiao. Wind Noise Reduction using Non-Negative Sparse Coding. In 2007
IEEE Workshop on Machine Learning for Signal Processing, pages 431436. IEEE, Aug. 2007. ISBN
978-1-4244-1565-6. doi: 10.1109/MLSP.2007.4414345. URL
http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4414345.
M. N. Schmidt, O. Winther, and L. K. Hansen. Bayesian non-negative matrix factorization. In Independent
Component Analysis and Signal Separation, pages 540547, 2009.
N. Seichepine, S. Essid, C. Fevotte, and O. Cappe. Soft nonnegative matrix co-factorization with application to
multimodal speaker diarization. In IEEE International Conference on Acoustics, Speech, and Signal Processing
(ICASSP), Vancouver, 2013.
N. Seichepine, S. Essid, C. Fevotte, and O. Cappe. Piecewise constant nonnegative matrix factorization. In IEEE
International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Florence, Italy, May 2014a.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

167 / 170

Conclusion

NMF software & bibliography

Bibliography XIII
N. Seichepine, S. Essid, C. Fevotte, and O. Cappe. Soft l1 and l2 coupling of nonnegative matrix factorization
problems. submitted to IEEE transactions on signal processing, 2014b.
A. Shashua and T. Hazan. Non-negative tensor factorization with applications to statistics and computer vision. In
ICML 2005: Proceedings of the 22nd International Conference on Machine Learning, pages 792799, 2005.
L. S. R. Simon and E. Vincent. A general framework for online audio source separation. In International conference
on Latent Variable Analysis and Signal Separation, Tel-Aviv, Israel, Mar. 2012.
J. Sivic, B. Russell, A. Efros, A. Zisserman, and W. Freeman. Discovering objets and their locations in images. In
ICCV, Beijing, China, 2005.
P. Smaragdis and J. C. Brown. Non-negative matrix factorization for polyphonic music transcription. In IEEE
Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), Oct. 2003.
D. Soukup and I. Bajla. Robust object recognition under partial occlusions using NMF. Computational intelligence
and neuroscience, 2008. URL http://www.hindawi.com/journals/cin/aip/857453/.
D. Sun and R. Mazumder. Non-negative matrix completion for bandwidth extension: A convex optimization
approach. Machine Learning for Signal . . . , 2013. URL
http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6661924.
Y. F. Tan and C. Fvotte. Automatic relevance determination in nonnegative matrix factorization with the
-divergence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35:15921605, 2013.
M. Trkan and C. Guillemot. Image prediction based on neighbor-embedding methods. IEEE Transactions on Image
Processing, 21(4):18851898, 2011.
E. Vincent, N. Bertin, and R. Badeau. Two nonnegative matrix factorization methods for polyphonic pitch
transcription. In Proc. Music Information Retrieval Evaluation eXchange (MIREX), 2007.
E. Vincent, N. Bertin, and R. Badeau. Harmonic and inharmonic Nonnegative Matrix Factorization for Polyphonic
Pitch transcription. In 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, pages
109112. IEEE, Mar. 2008. ISBN 978-1-4244-1483-3. doi: 10.1109/ICASSP.2008.4517558. URL
http://ieeexplore.ieee.org/articleDetails.jsp?arnumber=4517558.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

168 / 170

Conclusion

NMF software & bibliography

Bibliography XIV
E. Vincent, N. Bertin, and R. Badeau. Adaptive Harmonic Spectral Decomposition for Multiple Pitch Estimation.
IEEE Transactions on Audio, Speech, and Language Processing, 18(3):528537, Mar. 2010. ISSN 1558-7916.
doi: 10.1109/TASL.2009.2034186. URL
http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5282583.
A. Vinciarelli, A. Dielmann, S. Favre, and H. Salamin. Canal9: A database of political debates for analysis of social
interactions. In IEEE International Workshop on Social Signal Processing, Amsterdam, 2009. Ieee. ISBN
978-1-4244-4800-5. doi: 10.1109/ACII.2009.5349466. URL
http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5349466.
T. Virtanen. Monaural Sound Source Separation by Nonnegative Matrix Factorization With Temporal Continuity
and Sparseness Criteria. IEEE Transactions on Audio, Speech and Language Processing, 15(3):10661074, 2007.
T. Virtanen and A. Cemgil. Mixtures of gamma priors for non-negative matrix factorization based speech separation.
In Independent Component Analysis and Signal Separation. Springer-Verlag, 2009. URL
http://link.springer.com/chapter/10.1007/978-3-642-00599-2_81.
F. Wang, T. Li, X. Wang, S. Zhu, and C. Ding. Community discovery using nonnegative matrix factorization. Data
Mining and Knowledge Discovery, 22(3):493521, July 2010. ISSN 1384-5810. doi:
10.1007/s10618-010-0181-y. URL http://link.springer.com/10.1007/s10618-010-0181-y.
K. Wilson, B. Raj, and P. Smaragdis. Regularized non-negative matrix factorization with temporal dependencies for
speech denoising. In INTERSPEECH, 2008. URL http:
//www.cs.illinois.edu/~paris/Paris_Smaragdis_page/Paris_Smaragdis_Publications_files/wilson-is2008.pdf.
W. Xu, X. Liu, and Y. Gong. Document clustering based on non-negative matrix factorization. In Proceedings of
the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval SIGIR 03, page 267, New York, New York, USA, July 2003. ACM Press. ISBN 1581136463. doi:
10.1145/860435.860485. URL http://dl.acm.org/citation.cfm?id=860435.860485.
Z. Yang and E. Oja. Linear and nonlinear projective nonnegative matrix factorization. IEEE Trans. on Neural
Networks, 21:734749, 2010.
Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

169 / 170

Conclusion

NMF software & bibliography

Bibliography XV

Z. Yang and E. Oja. Unified Development of Multiplicative Algorithms for Linear and Quadratic Nonnegative Matrix
Factorization. IEEE Trans. Neural Networks, 22(12):18781891, 2011. doi:
http://dx.doi.org/10.1109/TNN.2011.2170094.
K. Yilmaz and A. T. Cemgil. Probabilistic latent tensor factorisation. In Proc. of International Conference on Latent
Variable analysis and Signal Separation, pages 346353, 2010.
K. Y. Yilmaz, A. T. Cemgil, and U. Simsekli. Generalized coupled tensor factorization. In NIPS, 2011.
R. Zdunek and A. Cichocki. Non-negative matrix factorization with quasi-Newton optimization. In Eighth
International Conference on Artificial Intelligence and Soft Computing (ICAISC), pages 870879, 2006.
M. Zetlaoui, M. Feinberg, P. Verger, and S. Clmenon. Extraction of food consumption systems by non-negative
matrix factorization (NMF) for the assessment of food choices. Technical report, Arxiv, 2010. URL
http://hal.archives-ouvertes.fr/docs/00/48/47/94/PDF/NMF_food.pdf.

Essid & Ozerov (TPT/Technicolor)

A tutorial on NMF

ICME 2014

170 / 170