Matthias Niessner (@MattNiessner)
2025-12-23 | โค๏ธ 546 | ๐ 83
๐ขPix2NPHM: Learning to Regress NPHM Reconstructions From a Single Image๐ข
We directly regress neural parametric head models (NPHMs) from a single image โ fast, stable, and significantly more expressive than classical 3DMMs such as FLAME.
Face tracking & 3D reconstruction are often limited by the representational capacity of PCA-based face models. By lifting NPHMs to a first-class reconstruction primitive, we enable more accurate geometry, richer expressions, and finer animation control.
Pix2NPHM obtains fast and reliable NPHM reconstructions on real-world data. Inference-time optimization against surface normals and canonical point maps can further increase fidelity.
Key to successful and generalized training of our ViT-based network are: (1) large-scale registration of existing 3D head datasets, and (2) self-supervised training on vast in-the-wild 2D video datasets using pseudo ground-truth surface normals.
Finally, we show that geometry-aware pretraining on pixel-aligned reconstruction tasks significantly outperforms generic visual pretraining (e.g., DINO-style features) in terms of generalization.
๐https://simongiebenhain.github.io/Pix2NPHM/ ๐ฅhttps://www.youtube.com/watch?v=MgpEJC5p1Ts&feature=youtu.be
Great work by @SGiebenhain, @TobiasKirschst1, @liamschoneveld, Davide Davoli, Zhe Chen
๐ ์๋ณธ ๋งํฌ
- https://simongiebenhain.github.io/Pix2NPHM/
- https://www.youtube.com/watch?v=MgpEJC5p1Ts&feature=youtu.be
๋ฏธ๋์ด
๐ Related
- thx-akhaliq-for-sharing-control-the-camera-control-the
- introducing-shaper-a-method-for-robust-conditional-3d-shape
- thinking-with-blender-meet-viga-a-multimodal-agent-that
- watch-the-awesome-4dgs-plugin-running-in-lichtfeld-studio
- you-could-direct-a-video-like-a-real-3d-world-researchers