SFDM: Robust Decomposition of Geometry and Reflectance for Realistic Face Rendering from Sparse-view Images

In this study, we introduce a novel two-stage technique for decomposing and reconstructing facial features from sparse-view images, a task made challenging by the unique geometry and complex skin reflectance of each individual. To synthesize 3D facial models more realistically, we endeavor to decouple key facial attributes from the RGB color, including geometry, diffuse reflectance, and specular reflectance. Specifically, we design a Sparse-view Face Decomposition Model (SFDM): 1) In the first stage, we create a general facial template from a wide array of individual faces, encapsulating essential geometric and reflectance characteristics. 2) Guided by this template, we refine a specific facial model for each individual in the second stage, considering the interaction between geometry and reflectance, as well as the effects of subsurface scattering on the skin. With these advances, our method can reconstruct high-quality facial representations from as few as three images. The comprehensive evaluation and comparison reveal that our approach outperforms existing methods by effectively disentangling geometric and reflectance components, significantly enhancing the quality of synthesized novel views, and paving the way for applications in facial relighting and reflectance editing.

# Create Conda Environment
conda create -n sfdm python=3.8 
conda activate sfdm

# Install requirements
pip install -r requirement.txt

We mainly use the FaceScape Dataset for evaluation. Please refer to NeuFace for instructions on preprocessing the FaceScape dataset and downloading the pretrained ImFace model.

The pretrained facial template can be downloaded at OneDrive

python tools/train.py --cfg ./config/train_sfdm_st2.yaml

# Evaluation
# change eval_type in the config file to 'evalset'
python tools/eval.py --cfg ./config/train_sfdm_st2.yaml --out_dir ./outputs/results --ckpt $PATH_TO_CKPT

# Evaluation
# change eval_type in the config file to 'testset'
python tools/eval.py --cfg ./config/train_sfdm_st2.yaml --out_dir ./outputs/results --ckpt $PATH_TO_CKPT

@inproceedings{jin2025sfdm,
  title={SFDM: Robust Decomposition of Geometry and Reflectance for Realistic Face Rendering from Sparse-view Images},
  author={Jin, Daisheng and Hu, Jiangbei and Xu, Baixin and Dai, Yuxin and Qian, Chen and He, Ying},
  booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference},
  pages={26409--26419},
  year={2025}
}

The codebase is developed based on NeuFace and 3dheads. Many thanks for these great projects!