Few-shot Hybrid Domain Adaptation of Image Generators

Hengjia Li*1, 4, Yang Liu*2, Linxuan Xia1, Yuqi Lin1, Wenxiao Wang3, Tu Zheng4, Zheng Yang4, Xiaohui Zhong5, Xiaobo Ren6, Xiaofei He1, 4,
1 State Key Lab of CAD&CG, Zhejiang University, 2Alibaba Cloud, 3 Zhejiang University, 4 Fabu Inc, 5 Ningbo Beilun Third Container Terminal Co., Ltd, 6 Ningbo Zhoushan Port Co., Ltd.
Paper arXiv Code

Abstract



Can a pre-trained generator be adapted to the hybrid of multiple target domains and generate images with integrated attributes of them? In this work, we introduce a new task -- Few-shot Hybrid Domain Adaptation (HDA). Given a source generator and several target domains, HDA aims to acquire an adapted generator that preserves the integrated attributes of all target domains, without overriding the source domain's characteristics. Compared with Domain Adaptation (DA), HDA offers greater flexibility and versatility to adapt generators to more composite and expansive domains. Simultaneously, HDA also presents more challenges than DA as we have access only to images from individual target domains and lack authentic images from the hybrid domain. To address this issue, we introduce a discriminator-free framework that directly encodes different domains' images into well-separable subspaces. To achieve HDA, we propose a novel directional subspace loss comprised of a distance loss and a direction loss. Concretely, the distance loss blends the attributes of all target domains by reducing the distances from generated images to all target subspaces. The direction loss preserves the characteristics from the source domain by guiding the adaptation along the perpendicular to subspaces. Experiments show that our method can obtain numerous domain-specific attributes in a single adapted generator, which surpasses the baseline methods in semantic similarity, image fidelity, and cross-domain consistency.

Method



Results on 10-shot HDA





Results on 3-domain HDA of sketch-smile-baby



Results on 10-shot DA





BibTeX

@article{li2023few,
  title={Few-shot Hybrid Domain Adaptation of Image Generators},
  author={Li, Hengjia and Liu, Yang and Xia, Linxuan and Lin, Yuqi and Zheng, Tu and Yang, Zheng and Wang, Wenxiao and Zhong, Xiaohui and Ren, Xiaobo and He, Xiaofei},
  journal={arXiv preprint arXiv:2310.19378},
  year={2023}
}