Taxonomy-Structured Domain Adaptation (TSDA)

This repo contains the code for our ICML 2023 paper:
Taxonomy-Structured Domain Adaptation
Tianyi Liu*, Zihao Xu*, Hao He, Guang-Yuan Hao, Guang-He Lee, Hao Wang
Fortieth International Conference on Machine Learning (ICML), 2023
[Paper] [OpenReview] [PPT] [Talk (Youtube)] [Talk (Bilibili)]
"*" indicates equal contribution.

Outline for This README

• Brief Introduction for TSDA
• Method Overview
• Theorem (Informal)
• Installation
• Code for Different Datasets
• Visualization of Experiment Results on DT-14
• Quantitative Results
• Domain Taxonomy
• Encoding Visualization
• Related Works
• Reference

Brief Introduction for TSDA

For classical domain adaptation methods such as DANN, they enforce uniform alignment to boost the generalization ability of models. However, recent studies have shown that, such uniform alignment can harm domain adaptation performance. To deal with this problem, we incorporate domain taxonomy into domain adaptation process. With domain taxonomy, we can break the uniform alignment in domain adaptation. The equilibrium recovers the classic adversarial domain adaptation’s solution if given a non-informative domain taxonomy (e.g., a flat taxonomy where all leaf nodes connect to the root node) while yielding non-trivial results with other taxonomies. See the following figure for an example.

An example of using domain taxonomy to break the uniform alignment. For our model, the middle representation for basset and Beagle should be more similar than the one for basset and tabby.

Method Overview

We build on the classic adversarial framework and introduce a novel taxonomist, which competes with the adversarial discriminator to preserve the taxonomy information. Encoder E, predictor F, discriminator D and taxonomist T play a min-max game. The discriminator tries to remove the domain-related information by enforcing uniform alignment on the encodings e, while the taxonomist tries to reconstruct the domain taxonomy based on the encodings. The balance between the discriminator and the taxonomist will make the encoding preserve necessary domain information. To facilitate such process, we transform the domain taxonomy to a domain distance matrix A and input it into the encoder. We highlight the key difference between the traditional model and our model in red.

Theorem (Informal, See Formal Definition in the Paper)

• The introduction of the taxonomist prevents the discriminator from enforcing uniform alignment.
• TSDA can recover DANN with a non-informative taxonomy (e.g., a flat taxonomy where all leaf nodes connect to the root node). Refer to the right Figure for an example of non-informative taxonomy.
• DANN with weighted pairwise discriminators can only produce uniform alignment.

Installation

conda create -n TSDA python=3.8
conda activate TSDA
conda install pytorch==1.11.0 torchvision==0.12.0 cudatoolkit=11.3 -c pytorch
pip install -r requirements.txt

Code for Different Datasets

In the directory of each dataset, there are detailed steps on how to train TSDA.

• Experiments on Toy Datasets
• Experiments on Real Datasets

Visualization of Experiment Results on DT-14

Quantitative Results

Toy Datasets: DT-14 and DT-40

ImageNet-Attribute-DT

CUB-DT

Domain Taxonomy

ImageNet-Attribute-DT

Domain taxonomy of ImageNet-Attribute-DT with 11 domains shown as leaf nodes. Note that leaf nodes are marked with black base and white text. Non-leaf nodes and their connection to leaf nodes are obtained from WordNet.

CUB-DT

Domain taxonomy of CUB-DT with 18 domains shown as leaf nodes. Note that leaf nodes are marked with black base and white text. For clarity we abbreviate domain names in the figure: Least Auklet (LA), Parakeet Auklet (PA), Red Winged Blackbird (RWB), Yellow Headed Blackbird (YHB), Lazuli Bunting (LB), Painted Bunting (PB), Bronzed Cowbird (BC), Shiny Cowbird (SC), Loggerhead Shrike (LS), Great Grey Shirke (GGS), Baird Sparrow (BS), Henslow's Sparrow (HS), Nelson's Sparrow (NS), Seaside Sparrow (SS), Canada Warbler (CW), Wilson's Warbler (WW), Chestnut-sided Warbler (CSW), Myrtle Warbler (MW).

Encoding Visualization

Imagnet-Attribute-DT

TSDA's learned encoding e for 11 domains on ImageNet-Attribute-DT. Domains related to "dogs", e.g., "Basset" and "Beagle", contain encodings in the middle, while domains related to "cats", e.g., "Tabby“, contain encodings on both sides; this is consistent with the domain taxonomy of Imagenet-Attribute-DT. Each domain contains 2 clusters because of the binary classification task.

CUB-DT

TSDA's learned encoding e for 18 domains on CUB-DT. Domain "Least Auklet" (LA) and domain "Parakeet Auklet" (PA) are very different from all others in the domain taxonomy of CUB-DT, correspondingly their encodings are on the far right side, with most of them in the bottom-right corner. Note that each domain has 2 clusters associated with the binary classification labels

Also Check Our Relevant Work

[1] Domain-Indexing Variational Bayes: Interpretable Domain Index for Domain Adaptation
Zihao Xu*, Guang-Yuan Hao*, Hao He, Hao Wang
Eleventh International Conference on Learning Representations, 2023
[Paper] [OpenReview] [PPT] [Talk (Youtube)] [Talk (Bilibili)]

[2] Graph-Relational Domain Adaptation
Zihao Xu, Hao He, Guang-He Lee, Yuyang Wang, Hao Wang
Tenth International Conference on Learning Representations (ICLR), 2022
[Paper] [Code] [Talk] [Slides]

[3] Continuously Indexed Domain Adaptation
Hao Wang*, Hao He*, Dina Katabi
Thirty-Seventh International Conference on Machine Learning (ICML), 2020
[Paper] [Code] [Talk] [Blog] [Slides] [Website]

Reference

Taxonomy-Structured Domain Adaptation

@inproceedings{TSDA,
  title={Taxonomy-Structured Domain Adaptation},
  author={Liu, Tianyi and Xu, Zihao and He, Hao and Hao, Guang-Yuan and Lee, Guang-He and Wang, Hao},
  booktitle={International Conference on on Machine Learning},
  year={2023}
}