naveen manwani (@NaveenManwani17)
2024-09-04 | ❤️ 81 | 🔁 9
🚨Paper Alert 🚨
➡️Paper Title: DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos
🌟Few pointers from the paper
🎯Despite significant advancements in monocular depth estimation for static images, estimating video depth in the open world remains challenging, since open-world videos are extremely diverse in content, motion, camera movement, and length.
🎯Authors in this paper presents “DepthCrafter”, an innovative method for generating temporally consistent long depth sequences with intricate details for open-world videos, without requiring any supplementary information such as camera poses or optical flow.
🎯 DepthCrafter achieves generalization ability to open-world videos by training a video-to-depth model from a pre-trained image-to-video diffusion model, through their meticulously designed three-stage training strategy with the compiled paired video-depth datasets.
🎯Their training approach enables the model to generate depth sequences with variable lengths at one time, up to 110 frames, and harvest both precise depth details and rich content diversity from realistic and synthetic datasets.
🎯 They also proposed an inference strategy that processes extremely long videos through segment-wise estimation and seamless stitching. Comprehensive evaluations on multiple datasets reveal that DepthCrafter achieves state-of-the-art performance in open-world video depth estimation under zero-shot settings.
🎯Furthermore, DepthCrafter facilitates various downstream applications, including depth-based visual effects and conditional video generation.
🏢Organization: @TencentGlobal AI Lab, @hkust , ARC Lab, Tencent PCG
🧙Paper Authors: @wbhu_cuhk , Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xiaodong Cun, Yong Zhang, Long Quan, @yshan2u
1️⃣Read the Full Paper here: https://arxiv.org/abs/2409.02095
2️⃣Project Page: https://depthcrafter.github.io/
3️⃣Code: Coming 🔜
🎥 Be sure to watch the attached Demo Video -Sound on 🔊🔊
Find this Valuable 💎 ?
♻️QT and teach your network something new
Follow me 👣, @NaveenManwani17 , for the latest updates on Tech and AI-related news, insightful research papers, and exciting announcements.
미디어
