Lipstick ain't enough:
Beyond Color-Matching for In-the-Wild Makeup Transfer

CVPR 2021
Thao Nguyen1
Anh Tran1,2
Minh Hoai1,3
1VinAI Research
2VinUniversity
3Stony Brook University
[Paper]
[GitHub]
[Datasets]
[Colab]
[Poster]
[Slides]

Star     Tweet It!
In-the-wild facial makeup consists of both color transfer and pattern addition. We propose a holistic method that can transfer the colors and patterns from a reference makeup style to another image.

Abstract

Makeup transfer is the task of applying on a source face the makeup style from a reference image. Real-life makeups are diverse and wild, which cover not only color-changing but also patterns, such as stickers, blushes, and jewelries. However, existing works overlooked the latter components and confined makeup transfer to color manipulation, focusing only on light makeup styles. In this work, we propose a holistic makeup transfer framework that can handle all the mentioned makeup components. It consists of an improved color transfer branch and a novel pattern transfer branch to learn all makeup properties, including color, shape, texture, and location. To train and evaluate such a system, we also introduce new makeup datasets for real and synthetic extreme makeup. Experimental results show that our framework achieves the state of the art performance on both light and extreme makeup styles.


In this work, we extend the definition of the makeup transfer.
Our proposed method can replicate both color and pattern (drawings, glitters, etc.) from target style.

Paper Videos

✌️ Two-Minute Video

🖐️ Five-Minute Video


Code and Datasets

You can try it in Google Colab here! Google Colab

[Github] [Datasets]


Paper and Supplementary Material

Thao Nguyen, Anh Tran, Minh Hoai.
Lipstick ain't enough: Beyond Color-Matching for In-the-Wild Makeup Transfer
In CVPR, 2021.
[paper] [supplementary] [poster]
BibTeX:
@inproceedings{m_Nguyen-etal-CVPR21,
      author = {Thao Nguyen and Anh Tran and Minh Hoai},
      title = {Lipstick ain't enough: Beyond Color Matching for In-the-Wild Makeup Transfer},
      year = {2021},
      booktitle = {Proceedings of the {IEEE} Conference on Computer Vision and Pattern Recognition (CVPR)}
   }


👁 Qualitative Performane Comparisons 👁

We show additional results on multiple datasets: Makeup Transfer (BeautyGAN), CPM-Synt-1, CPM-Synt-2, and CPM-Real.
Baselises (left to right): DMT (arXiv 2019), BeautyGAN (ACM'MM 2018), LADN (ICCV 2019), and PSGAN (CVPR 2020).


✓ Additional results on Makeup Transfer Dataset
✓ Additional results on CPM-Synt-1
✓ Additional results on CPM-Synt-2
✓ Additional results on CPM-Real
Difficult Cases
Other applications (interpolation and partial makeup) can be found in our main paper or supplementary.
We skipped some baselines which have no released model, such as BeautyGLOW and CA-GAN.
Further info: ⇨ [awesome-makeup-transfer]: a curated list of Makeup Transfer and related resources.


Acknowledgements

This template was originally made by Phillip Isola and Richard Zhang for a colorful ECCV project; the code can be found here. Thank you (◍•ڡ•◍).
▶ thaoshibe.github.io's clustrmaps 🌎.