Identity preserving face completion for large ocular region occlusion

Yajie Zhao, Weikai Chen, Jun Xing, Xiaoming Li, Zach Bessinger, Fuchang Liu, Wangmeng Zuo, Ruigang Yang

Research output: Contribution to conferencePaperpeer-review

4 Scopus citations

Abstract

We present a novel deep learning approach to synthesize complete face images in the presence of large ocular region occlusions. This is motivated by recent surge of VR/AR displays that hinder face-to-face communications. Different from the state-of-the-art face inpainting methods that have no control over the synthesized content and can only handle frontal face pose, our approach can faithfully recover the missing content under various head poses while preserving the identity. At the core of our method is a novel generative network with dedicated constraints to regularize the synthesis process. To preserve the identity, our network takes an arbitrary occlusion-free image of the target identity to infer the missing content, and its high-level CNN features as an identity prior to regularize the searching space of generator. Since the input reference image may have a different pose, a pose map and a novel pose discriminator are further adopted to supervise the learning of implicit pose transformations. Our method is capable of generating coherent facial inpainting with consistent identity over videos with large variations of head motions. Experiments on both synthesized and real data demonstrate that our method greatly outperforms the state-of-the-art methods in terms of both synthesis quality and robustness.

Original languageEnglish
StatePublished - 2019
Event29th British Machine Vision Conference, BMVC 2018 - Newcastle, United Kingdom
Duration: Sep 3 2018Sep 6 2018

Conference

Conference29th British Machine Vision Conference, BMVC 2018
Country/TerritoryUnited Kingdom
CityNewcastle
Period9/3/189/6/18

Bibliographical note

Publisher Copyright:
© 2018. The copyright of this document resides with its authors.

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition

Fingerprint

Dive into the research topics of 'Identity preserving face completion for large ocular region occlusion'. Together they form a unique fingerprint.

Cite this