0% found this document useful (0 votes)
4 views73 pages

AML - Lecture - 12 - 29nov24

The document discusses advanced machine learning concepts, particularly focusing on domain adaptation and the challenges posed by domain shifts in visual recognition systems. It outlines various adaptation settings, algorithms, and recent advancements in the field, including adversarial domain adaptation and generative approaches. Additionally, it highlights the importance of synthetic data and new benchmark datasets for improving model performance across different domains.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
4 views73 pages

AML - Lecture - 12 - 29nov24

The document discusses advanced machine learning concepts, particularly focusing on domain adaptation and the challenges posed by domain shifts in visual recognition systems. It outlines various adaptation settings, algorithms, and recent advancements in the field, including adversarial domain adaptation and generative approaches. Additionally, it highlights the importance of synthetic data and new benchmark datasets for improving model performance across different domains.
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 73

AM ML

Advanced Machine Learning

A.A. 2024/2025
Tatiana Tommasi
1
Domain Adaptation

2
On the success of Deep Learning models...

Is there a bird?

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 3


On the success of Deep Learning models...

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 4


And failure...

https://bam-dataset.org/

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 5


And failure...

https://bam-dataset.org/

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 6


Domain Shift
Visual appearance changes degrade the performance of visual
recognition systems.

Accuracy: Accuracy:
85% 55%

B. Kulis, K. Saenko, and T. Darrell, "What You Saw is Not What You Get: Domain Adaptation Using Asymmetric Kernel Transforms" CVPR, 2011.

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 7


Domain Shift

Appearance changes due to seasonal and time changes.

Z. Chen et al., "Deep Learning Features at Scale for Visual Place Recognition". ICRA 2017.

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 8


Domain Shift
Appearance changes due to illumination conditions

S. Hwang et al., “Multispectral Pedestrian Detection: Benchmark Dataset and Baseline”, CVPR, 2015.

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 9


Domain Shift

Appearance changes due to different sensors.

S. Hwang et al., “Multispectral Pedestrian Detection: Benchmark Dataset and Baseline”, CVPR, 2015.

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 10


Domain Shift

Overcoming costly/unfeasible data collection

https://www.zdnet.com/article/robots-to-the-rescue-searching-for-survivors-checking-on-structural-damage-in-japan/

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 11


Domain Shift

Use of synthetic data

http://synthia-dataset.net/

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 12


Domain Shift and Dataset Bias

Torralba and Efros,”Unbiased look at dataset bias, CVPR, 2011.

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 13


A bit of Notation

Feature space
Output space
Input variable
Output variable

Domain

Task

14
The Domain Adaptation Problem

Source Domain Target Domain

DA problem

15
The Domain Adaptation Problem

Source Domain Target Domain

DA problem

covariate shift assumption


16
The Domain Adaptation Settings

Unsupervised Domain Adaptation


Labelled Source Domain Unlabelled target Domain

dog elephant

house

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 17


The Domain Adaptation Settings

Unsupervised Domain Adaptation


Labelled Source Domain Unlabelled target Domain

dog elephant

house

train
Transductive
Setting test
Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 18
Variants: number of sources
Single Multi

elephant dog dog elephant


house
dog elephant elephant

house house dog


house
Mixed

elephant

elephant elephant house


dog

house house dog

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 19


If you want to know more about the DA settings
Variants: label overlap https://europe.naverlabs.com/eccv-2020-domain-adaptation-tutorial/

Closed Set Open Set

Partial
dog elephant dog
elephant

elephant

house
dog

house
Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 20
Without target data!

Domain Generalization Continuous DA

dog elephant

house

21
Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019
First visual testbed dataset

Office 31

Saenko, K., et al. “Adapting visual category models to new domains.” In ECCV 2010.
Gong, B., et al. “Geodesic flow kernel for unsupervised domain adaptation.” In CVPR 2012. 22
Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019
First (Shallow) Algorithms

Transfer Component Analysis


Learn a low rank empirical
kernel mapping

Pan, S., et al. . “Domain Adaptation via Transfer Component Analysis”. In TNN. 2011.
F. Orabona & T. Tommasi. Tutorial on Domain Adaptation and Transfer Learning. In ECCV 2014. 23
First (Shallow) Algorithms

Maximum Mean Discrepancy


Distance between embeddings of the probability
distributions in a reproducing kernel Hilbert space.

Gretton, A., et al. . “A kernel method for the two-sample-problem”. In NIPS. 2007.
F. Orabona & T. Tommasi. Tutorial on Domain Adaptation and Transfer Learning. In ECCV 2014. 24
First (Shallow) Algorithms

[Gopalan et al. ICCV


2011 ]

25
F. Orabona & T. Tommasi. Tutorial on Domain Adaptation and Transfer Learning. In ECCV 2014.
First (Shallow) Algorithms

Geodesic Flow Kernel [Gong et al. CVPR 2012]

26
F. Orabona & T. Tommasi. Tutorial on Domain Adaptation and Transfer Learning. In ECCV 2014.
First (Shallow) Algorithms

Long, M., et al. “Learning Transferable Features with Deep Adaptation Networks”. In ICML 2015.. 27
What happens when moving to deep features?

Long, M., et al. “Learning Transferable Features with Deep Adaptation Networks”. In ICML 2015.. 28
Deep Features = Universal Representation?

SURF DeCAF6

29
Donahue, J., et al. (2014). Decaf: A deep convolutional activation feature for generic visual recognition. In ICML.
2019/20 30
Long, M., et al. “Learning Transferable Features with Deep Adaptation Networks”. In ICML 2015..
31
Long, M., et al. “Learning Transferable Features with Deep Adaptation Networks”. In ICML 2015..
Office-31 (2017)

32
Long, M., et al. “Deep transfer learning with joint adaptation networks”. In ICML 2017.
Office-31 (2017)

33
Long, M., et al. “Deep transfer learning with joint adaptation networks”. In ICML 2017.
Office-31 (2017)

34
Adversarial Domain Adaptation (RevGrad)

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019


Ganin, Y., et al. “Domain-adversarial training of neural networks”. The Journal of Machine Learning Research, 35
17(1), 2096-2030, 2016.
Adversarial Domain Adaptation (RevGrad)

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019


36
Adversarial Domain Adaptation (RevGrad)

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019


37
Adversarial Domain Adaptation (RevGrad)

Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019


38
Adversarial Domain Adaptation (RevGrad-DANN)

Ganin, Y., et al. “Domain-adversarial training of neural networks”. The Journal of Machine Learning
Research, 17(1), 2096-2030, 2016. 39
Adversarial Domain Adaptation (RevGrad-DANN)

Ganin, Y., et al. “Domain-adversarial training of neural networks”. The Journal of Machine Learning
Research, 17(1), 2096-2030, 2016. 40
Adversarial Domain Adaptation (RevGrad-DANN)

“Unsupervised Domain Adaptation by Backpropagation”. ICML 2015

41
Adversarial Domain Adaptation (RevGrad-DANN)

Ganin, Y., et al. “Domain-adversarial training of neural networks”. The Journal of Machine Learning
Research, 17(1), 2096-2030, 2016. 42
Adversarial Domain Adaptation (RevGrad-DANN)

Ganin, Y., et al. “Domain-adversarial training of neural networks”. The Journal of Machine Learning
Research, 17(1), 2096-2030, 2016. 43
Asymmetric Adversarial Domain Adaptation

https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123690579.pdf
44
Adversarial Domain Adaptation (ADDA)

Tzeng, E., Hoffman, J., Saenko, K., & Darrell, T., ” Adversarial discriminative domain adaptation”. In CVPR 2017. 45
Adversarial Domain Adaptation (ADDA)

46
Adversarial Domain Adaptation (ADDA)

47
New Benchmark Datasets: PACS

48
http://www.eecs.qmul.ac.uk/~dl307/project_iccv2017
New Benchmark Datasets: Office-Home

49
http://hemanthdv.org/OfficeHome-Dataset/ 49
New Benchmark Datasets: DomainNet

50
https://github.com/VisionLearningGroup/visda-2019-public/tree/master/multisource
New Benchmark Datasets: Semantic Segmentation

Tsai, Y. H. et al. “Learning to adapt structured output space for semantic segmentation”. In CVPR 2018.
51
Pixel-Level DA (Generative Approaches)

https://skymind.ai/wiki/generative-adversarial-network-gan
52
Pixel-Level DA (Generative Approaches)

Bousmalis, K., Silberman, N., Dohan, D., Erhan, D., & Krishnan, D., “Unsupervised pixel-level domain
adaptation with generative adversarial networks”. In CVPR 2017. 53
Unpaired Image-to-Image Translation: Cycle-GAN

Zhu, J. Y., Park, T., Isola, P., & Efros, A.. “Unpaired image-to-image translation using cycle-consistent adversarial
54
networks”. In ICCV 2017.
Cycle-GAN

Zhu, J. Y., Park, T., Isola, P., & Efros, A.. “Unpaired image-to-image translation using cycle-consistent adversarial 55
networks”. In ICCV 2017.
SBADA-GAN

Russo, P., Carlucci, F. M., Tommasi, T., & Caputo, B. “From source to target and back: symmetric bi-directional adaptive gan”. In CVPR 2018.

56
SBADA-GAN

57
Pixel-Level DA in Detection

http://openaccess.thecvf.com/content_cvpr_2018/papers/Inoue_Cross-Domain_Weakly-Supervised_Object_CVPR_2018_paper.pdf
58
Pixel-Level DA in Detection

http://openaccess.thecvf.com/content_CVPR_2019/papers/Kim_Diversify_and_Match_A_Domain_Adaptive_Representation_Lear
ning_Paradigm_for_CVPR_2019_paper.pdf
59
Batch-Normalization for DA

FC

BN

ReLU

FC

BN

ReLU Ioffe, S., & Szegedy, C. “Batch Normalization: Accelerating Deep Network Training by Reducing Internal
Covariate Shift: Automatic Domain Alignment Layers”. ICML 2015.
http://cs231n.github.io/neural-networks-2/#batchnorm 60
Batch-Normalization for DA

FC

BN

ReLU

FC

BN

ReLU Ioffe, S., & Szegedy, C. “Batch Normalization: Accelerating Deep Network Training by Reducing Internal
Covariate Shift: Automatic Domain Alignment Layers”. ICML 2015.
http://cs231n.github.io/neural-networks-2/#batchnorm 61
Batch-Normalization for DA

dog elephant

house

Carlucci, F. M., Porzi, L., Caputo, B., Ricci, E., & Rota Bulò, S. “AutoDIAL: Automatic Domain Alignment Layers”. ICCV 2017.
Carlucci, F. M., Porzi, L., Caputo, B., Ricci, E., & Rota Bulò, S. “Just dial: Domain alignment layers for unsupervised domain 62
adaptation”. ICIAP 2017.
Multi-Source Batch-Normalization for DA
w
domain assignment

elephant
elephan
t
h u e
elephant o s
do
g

house dog
house

Mancini, M., Porzi, L., Rota Bulò, S., Caputo, B., & Ricci, E. “Boosting Domain Adaptation by Discovering 63
Latent Domains”. CVPR 2018.
Multi-Source Batch-Normalization for DA

64
Larger Norm More Transferrable

https://openaccess.thecvf.com/content_ICCV_2019/papers/Xu_Larger_Norm_More_Transferable_An_A
daptive_Feature_Norm_Approach_for_ICCV_2019_paper.pdf
65
Domain Generalization

66
Self-Challenging for Domain Generalization

https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123470120.pdf 67
Jigsaw Puzzle Based Domain Generalization

http://openaccess.thecvf.com/content_CVPR_2019/html/Carlucci_Domain_Generalization_by_Solving_Jigsaw_Puzzles_CVP 68
R_2019_paper.html
Jigsaw Puzzle Based Domain Generalization

http://openaccess.thecvf.com/content_CVPR_2019/html/Carlucci_Domain_Generalization_by_Solving_Jigsaw_Puzzles_CVPR_20 69
19_paper.html
Jigsaw Puzzle Based Domain Generalization

70
Jigsaw Puzzle Based Domain Generalization

71
Jigsaw Puzzle Based Domain Generalization

72
Jigsaw Puzzle Based Domain Generalization

73

You might also like