Minghuan Liu (@ericliuof97)
2025-02-17 | โค๏ธ 166 | ๐ 29
Introducing our recent progress to utilize synthetic dataโRE3SIMโa Real-to-Sim-to-Real pipeline that integrates Gaussian splatting with NVIDIA Isaac Simโs PhysX engine, improving scene reconstruction and sim-to-real transfer for robotic manipulation tasks. Project: https://xshenhan.github.io/Re3Sim/
Highlights:
- High-fidelity geometry and vision: small sim-to-real gaps in both geometry and visual aspects.
- Highly efficient data collection: scene reconstruction in ~2.5 minutes and simulation data at 100 episodes per 10 minutes.
- Zero-shot sim-to-real transfer: limited simulation data brings high success rates.
Key Observation:
- Scaling law: Increasing the simulation data scale can enhance the success rate until it converges at a high-performance level.
- Mixing Sim-Real: Co-training real-world data can integrate the characteristics of both datasets.
This work was done by our talented undergraduate @xshenhan , with me, @yilun95, Junqiu Yu, Xiaoyang Lyu, Yang Tian, Bolun Wang, Weinan Zhang, and @pangjiangmiao !
๐ ์๋ณธ ๋งํฌ
๋ฏธ๋์ด
๐ Related
Auto-generated - needs manual review
Tags
domain-vision-3d domain-robotics domain-simulation domain-visionos