Tobias Kirschstein (@TobiasKirschst1)
2025-02-28 | โค๏ธ 166 | ๐ 28
Very excited to announce Avat3r! From just 4 input images, we directly construct an animatable 3D head avatar in a single forward pass.
This greatly simplifies avatar creation: The whole process of taking phone pictures to the final avatar can be done in minutes on simple GPU
๐ Related
Auto-generated - needs manual review
์ธ์ฉ ํธ์
Matthias Niessner (@MattNiessner)
๐ข๐ข ๐๐ฏ๐๐ญ๐๐ซ ๐ข๐ข
Avat3r creates high-quality 3D head avatars from just a few input images in a single forward pass with a new dynamic 3DGS reconstruction model.
Video: https://t.co/MOYmMZJnWJ Project: https://t.co/t9HBG1Lvcs
Our core idea is to make Gaussian Reconstruction Models animatable. We find that a simple cross-attention to an expression code sequence is already sufficient to model complex facial expressions. We then incorporate position maps from DUSt3R and feature maps from Sapiens to facilitate the prediction task. While DUSt3Rโs position maps act as a pixel-aligned initialization for the Gaussiansโ positions, the Sapiens feature maps help the cross-view transformer to match corresponding image tokens in the 4 input images.
One major challenge in creating a 3D head avatar from smartphone images comes from inconsistent facial expressions when the subject could not remain perfectly static during the capture. We eliminate this static requirement by simply showing our model input images with different facial expressions during training. This technique makes our model robust to inconsistent input images later on.
Finally, we show that despite the model has been trained with 4 input images, one can even create a 3D head avatar when only a single image is available. To achieve this, we employ a pre-trained 3D GAN to lift the single image to 3D and then render the 4 input images for our model. This allows us to create 3D head avatars from single images and even highly out-of-distribution examples like AI generated faces, paintings or statues.
Great work by @TobiasKirschst1 from his internship at Meta with Javier Romero, @ASevastopolsky, and @psyth91
๐ฌ ์์
