AML - Lecture - 12 - 29nov24
AML - Lecture - 12 - 29nov24
A.A. 2024/2025
Tatiana Tommasi
1
Domain Adaptation
2
On the success of Deep Learning models...
Is there a bird?
https://bam-dataset.org/
https://bam-dataset.org/
Accuracy: Accuracy:
85% 55%
B. Kulis, K. Saenko, and T. Darrell, "What You Saw is Not What You Get: Domain Adaptation Using Asymmetric Kernel Transforms" CVPR, 2011.
Z. Chen et al., "Deep Learning Features at Scale for Visual Place Recognition". ICRA 2017.
S. Hwang et al., “Multispectral Pedestrian Detection: Benchmark Dataset and Baseline”, CVPR, 2015.
S. Hwang et al., “Multispectral Pedestrian Detection: Benchmark Dataset and Baseline”, CVPR, 2015.
https://www.zdnet.com/article/robots-to-the-rescue-searching-for-survivors-checking-on-structural-damage-in-japan/
http://synthia-dataset.net/
Feature space
Output space
Input variable
Output variable
Domain
Task
14
The Domain Adaptation Problem
DA problem
15
The Domain Adaptation Problem
DA problem
dog elephant
house
dog elephant
house
train
Transductive
Setting test
Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 18
Variants: number of sources
Single Multi
elephant
Partial
dog elephant dog
elephant
elephant
house
dog
house
Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019 20
Without target data!
dog elephant
house
21
Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019
First visual testbed dataset
Office 31
Saenko, K., et al. “Adapting visual category models to new domains.” In ECCV 2010.
Gong, B., et al. “Geodesic flow kernel for unsupervised domain adaptation.” In CVPR 2012. 22
Slide credit: Mancini, Morerio, Tutorial at ICIAP 2019
First (Shallow) Algorithms
Pan, S., et al. . “Domain Adaptation via Transfer Component Analysis”. In TNN. 2011.
F. Orabona & T. Tommasi. Tutorial on Domain Adaptation and Transfer Learning. In ECCV 2014. 23
First (Shallow) Algorithms
Gretton, A., et al. . “A kernel method for the two-sample-problem”. In NIPS. 2007.
F. Orabona & T. Tommasi. Tutorial on Domain Adaptation and Transfer Learning. In ECCV 2014. 24
First (Shallow) Algorithms
25
F. Orabona & T. Tommasi. Tutorial on Domain Adaptation and Transfer Learning. In ECCV 2014.
First (Shallow) Algorithms
26
F. Orabona & T. Tommasi. Tutorial on Domain Adaptation and Transfer Learning. In ECCV 2014.
First (Shallow) Algorithms
Long, M., et al. “Learning Transferable Features with Deep Adaptation Networks”. In ICML 2015.. 27
What happens when moving to deep features?
Long, M., et al. “Learning Transferable Features with Deep Adaptation Networks”. In ICML 2015.. 28
Deep Features = Universal Representation?
SURF DeCAF6
29
Donahue, J., et al. (2014). Decaf: A deep convolutional activation feature for generic visual recognition. In ICML.
2019/20 30
Long, M., et al. “Learning Transferable Features with Deep Adaptation Networks”. In ICML 2015..
31
Long, M., et al. “Learning Transferable Features with Deep Adaptation Networks”. In ICML 2015..
Office-31 (2017)
32
Long, M., et al. “Deep transfer learning with joint adaptation networks”. In ICML 2017.
Office-31 (2017)
33
Long, M., et al. “Deep transfer learning with joint adaptation networks”. In ICML 2017.
Office-31 (2017)
34
Adversarial Domain Adaptation (RevGrad)
Ganin, Y., et al. “Domain-adversarial training of neural networks”. The Journal of Machine Learning
Research, 17(1), 2096-2030, 2016. 39
Adversarial Domain Adaptation (RevGrad-DANN)
Ganin, Y., et al. “Domain-adversarial training of neural networks”. The Journal of Machine Learning
Research, 17(1), 2096-2030, 2016. 40
Adversarial Domain Adaptation (RevGrad-DANN)
41
Adversarial Domain Adaptation (RevGrad-DANN)
Ganin, Y., et al. “Domain-adversarial training of neural networks”. The Journal of Machine Learning
Research, 17(1), 2096-2030, 2016. 42
Adversarial Domain Adaptation (RevGrad-DANN)
Ganin, Y., et al. “Domain-adversarial training of neural networks”. The Journal of Machine Learning
Research, 17(1), 2096-2030, 2016. 43
Asymmetric Adversarial Domain Adaptation
https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123690579.pdf
44
Adversarial Domain Adaptation (ADDA)
Tzeng, E., Hoffman, J., Saenko, K., & Darrell, T., ” Adversarial discriminative domain adaptation”. In CVPR 2017. 45
Adversarial Domain Adaptation (ADDA)
46
Adversarial Domain Adaptation (ADDA)
47
New Benchmark Datasets: PACS
48
http://www.eecs.qmul.ac.uk/~dl307/project_iccv2017
New Benchmark Datasets: Office-Home
49
http://hemanthdv.org/OfficeHome-Dataset/ 49
New Benchmark Datasets: DomainNet
50
https://github.com/VisionLearningGroup/visda-2019-public/tree/master/multisource
New Benchmark Datasets: Semantic Segmentation
Tsai, Y. H. et al. “Learning to adapt structured output space for semantic segmentation”. In CVPR 2018.
51
Pixel-Level DA (Generative Approaches)
https://skymind.ai/wiki/generative-adversarial-network-gan
52
Pixel-Level DA (Generative Approaches)
Bousmalis, K., Silberman, N., Dohan, D., Erhan, D., & Krishnan, D., “Unsupervised pixel-level domain
adaptation with generative adversarial networks”. In CVPR 2017. 53
Unpaired Image-to-Image Translation: Cycle-GAN
Zhu, J. Y., Park, T., Isola, P., & Efros, A.. “Unpaired image-to-image translation using cycle-consistent adversarial
54
networks”. In ICCV 2017.
Cycle-GAN
Zhu, J. Y., Park, T., Isola, P., & Efros, A.. “Unpaired image-to-image translation using cycle-consistent adversarial 55
networks”. In ICCV 2017.
SBADA-GAN
Russo, P., Carlucci, F. M., Tommasi, T., & Caputo, B. “From source to target and back: symmetric bi-directional adaptive gan”. In CVPR 2018.
56
SBADA-GAN
57
Pixel-Level DA in Detection
http://openaccess.thecvf.com/content_cvpr_2018/papers/Inoue_Cross-Domain_Weakly-Supervised_Object_CVPR_2018_paper.pdf
58
Pixel-Level DA in Detection
http://openaccess.thecvf.com/content_CVPR_2019/papers/Kim_Diversify_and_Match_A_Domain_Adaptive_Representation_Lear
ning_Paradigm_for_CVPR_2019_paper.pdf
59
Batch-Normalization for DA
FC
BN
ReLU
FC
BN
ReLU Ioffe, S., & Szegedy, C. “Batch Normalization: Accelerating Deep Network Training by Reducing Internal
Covariate Shift: Automatic Domain Alignment Layers”. ICML 2015.
http://cs231n.github.io/neural-networks-2/#batchnorm 60
Batch-Normalization for DA
FC
BN
ReLU
FC
BN
ReLU Ioffe, S., & Szegedy, C. “Batch Normalization: Accelerating Deep Network Training by Reducing Internal
Covariate Shift: Automatic Domain Alignment Layers”. ICML 2015.
http://cs231n.github.io/neural-networks-2/#batchnorm 61
Batch-Normalization for DA
dog elephant
house
Carlucci, F. M., Porzi, L., Caputo, B., Ricci, E., & Rota Bulò, S. “AutoDIAL: Automatic Domain Alignment Layers”. ICCV 2017.
Carlucci, F. M., Porzi, L., Caputo, B., Ricci, E., & Rota Bulò, S. “Just dial: Domain alignment layers for unsupervised domain 62
adaptation”. ICIAP 2017.
Multi-Source Batch-Normalization for DA
w
domain assignment
elephant
elephan
t
h u e
elephant o s
do
g
house dog
house
Mancini, M., Porzi, L., Rota Bulò, S., Caputo, B., & Ricci, E. “Boosting Domain Adaptation by Discovering 63
Latent Domains”. CVPR 2018.
Multi-Source Batch-Normalization for DA
64
Larger Norm More Transferrable
https://openaccess.thecvf.com/content_ICCV_2019/papers/Xu_Larger_Norm_More_Transferable_An_A
daptive_Feature_Norm_Approach_for_ICCV_2019_paper.pdf
65
Domain Generalization
66
Self-Challenging for Domain Generalization
https://www.ecva.net/papers/eccv_2020/papers_ECCV/papers/123470120.pdf 67
Jigsaw Puzzle Based Domain Generalization
http://openaccess.thecvf.com/content_CVPR_2019/html/Carlucci_Domain_Generalization_by_Solving_Jigsaw_Puzzles_CVP 68
R_2019_paper.html
Jigsaw Puzzle Based Domain Generalization
http://openaccess.thecvf.com/content_CVPR_2019/html/Carlucci_Domain_Generalization_by_Solving_Jigsaw_Puzzles_CVPR_20 69
19_paper.html
Jigsaw Puzzle Based Domain Generalization
70
Jigsaw Puzzle Based Domain Generalization
71
Jigsaw Puzzle Based Domain Generalization
72
Jigsaw Puzzle Based Domain Generalization
73