GitHub - OpenDriveLab/WorldEngine: WorldEngine: Towards the Era of Post-Training for Physical AI WorldEngine: Towards the Era of Post-Training for Physical AI - OpenDriveLab/WorldEngine

🌍 WorldEngine: Towards the Era of Post-Training for Physical AI

🎯 A post-training framework for Physical AI that systematically addresses the long-tail safety-critical data scarcity problem in autonomous driving.

Github: github.com/OpenDriveLab...
Project Page: opendrivelab.com/WorldEngine/

【MM-Hand 1.0】An open-source low-cost dexterous hand. Selected applicants will receive beta-version hardware in early 2026 at cost price (~USD $1,400 but tax not included) and direct engineering help.

🔗 Details: mmlab.hk/research/MM-...

Why do VLN agents need "step-by-step" language instructions? In the real world, humans don't give lengthy and detailed manuals; we give a intent.
📍 Our latest work, SparseVideoNav, tackles the "myopia" of LLM-based agents.

Website: opendrivelab.com/SparseVideoN...

We’ll be live in 1️⃣ hour: www.youtube.com/live/MWKQqKC...

[5/5] Bottom Line

• Not all robot data is equally valuable
• Fast iteration > bruteforce scaling
• Weight-space merging can outperform joint training
• Stage-aware advantage estimation helps long-horizon tasks

📄 Full report: Q1 2026
📦 Data + checkpoints + challenge: 2026

[4/5] Problem: Long-Horizon Credit Assignment

6-minute tasks. Which actions actually helped?

Solution → Stage Advantage:
• Decompose into semantic stages
• Predict advantage directly (not value-diff)
• Smoother supervision, less error compounding

[3/5] Problem: Expensive Iteration

Collect new data → Retrain everything → Repeat

Slow yet expensive.

How? Model Arithmetic:
• Train only on new data
• Merge via weight interpolation
• Merged model > full-dataset model

Models trained separately preserve distinct modes.

[2/5] Problem: Distribution Mismatch

Training data ≠ Model behavior ≠ Real-world execution

This gap causes failures.

Solution → Mode Consistency:
• DAgger for failure recovery
• Augmentation for coverage
• Inference smoothing for clean execution

🧥 Live-stream robotic teamwork that folds clothes. 6 clothes in 3 minutes straight.

χ₀ = 20hrs data + 8 A100s + 3 key insights:
- Mode Consistency: align your distributions
- Model Arithmetic: merge, don't retrain
- Stage Advantage: pivot wisely

🔗 mmlab.hk/research/kai0 checkout 3mins demo

In χ0, we aim to conquer the "Mount Everest" of robotics: 100% reliability in real-world garment manipulation. Tech Blog: mmlab.hk/research/kai0

24-Hour Live Demo at www.youtube.com/@OpenDriveLab, starting at 21:00 (UTC+8) on December 24

Imitation learning alone can't get robots to human-level intelligence and dexterity. In our new perspective, we argue that robots must learn from their own experience - a process we call self-directed learning.

Check it out ! openreview.net/forum?id=Seb...

🏟️But we’re not done yet - our workshop continues at #ICCV2025! And the challenge moves forward too, with more prizes and exciting updates. opendrivelab.com/challenge2025/

📹Our #CVPR2025 workshop and tutorial recordings are now online! Big thanks to our incredible speakers! Watch all the sessions here
🔗 Workshop: youtube.com/playlist?lis...
🔗 Tutorial: youtube.com/playlist?lis...

#ICCV2025 DetAny3D: Detect Anything 3D in the Wild

Can your 3D detector handle novel objects & unseen cameras from just a single image? DetAny3D can.

📜 Paper: arxiv.org/abs/2504.07958
💻 Code: github.com/OpenDriveLab...

🚀The AgiBot World Challenge @ IROS2025 starts now! More details on opendrivelab.com/challenge202...

⏰ Key Dates
Online Challenge: June 25th to September 1st
Grand Finals: October 19th in Hangzhou

How can we reliably simulate future driving scenarios under a wide range of ego driving behaviors? Check out: ReSim: Reliable World Simulation for Autonomous Driving
resim-world-model.github.io

🤔 How to reliably simulate future driving scenarios under a wide range of ego behaviors?

😎 Key ingredient: Co-training the world model on heterogeneous data, including real-world data with expert actions and simulated data with non-expert behaviors.

See ReSim: arxiv.org/abs/2506.09981

We are pleased to introduce FreeTacMan, a human-centric and robot-free visuo-tactile data collection system for high-quality and efficient robot manipulation!

🚀 Website: opendrivelab.com/FreeTacMan
📜 Paper: arxiv.org/abs/2506.01941
💻 Repo: github.com/OpenDriveLab...

🚀 Ready for the #IROS2025 challenge? We've got you covered!

🌍 Two identical sessions will run for different time zones. Don't miss it!

[Asia/Europe] 28 May 2025, 17:00 (UTC+8) us06web.zoom.us/j/88402705842
[America] 28 May 2025, 18:00 (UTC-7) us06web.zoom.us/j/86342172313

Computers Take the Wheel - HKU Bulletin The technology behind embodied AI and autonomous cars has made huge strides in recent times, helped in part by a system devised by HKU Professor Li Hongyang and his collaborators.

🚗 The technology behind embodied AI and autonomous cars has recently made huge strides. But how do computers take the wheel?

Check out our newest article at bulletin.hku.hk/cover-story-...

Forget slow autoregression and skip rigid full-sequence denoising! Nexus is a next-gen predictive pipeline for realistic, safety-critical driving scene generation.

Page: lnkd.in/gP4EuRFz
Paper: lnkd.in/gE7Svj-3
Repo: lnkd.in/gxuH959n

🤖 Spoiler alert!

👀 At IROS 2025 in Hangzhou, robots will compete, live on stage.

🔗 Details will be released soon (very soon) on opendrivelab.com/challenge202...

⏰ The #CVPR2025 submission deadline is approaching fast — don't forget to upload your results by May 10th!

🤖 Coming in early May: The AgiBot World Challenge! We are going into the real world — deploying them on real robots, live at #IROS2025.

🔗 Learn more at opendrivelab.com/challenge2025

Introducing AgiBot World Colosseo:
A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems. opendrivelab.com/blog/agibot-...

🌟 Previewing the UniAD 2.0

🚀 A milestone upgrade on the codebase of the #CVPR2023 best paper UniAD.

👉 Check out this branch github.com/OpenDriveLab..., and we will get you more details soon

🚀 This year, we’re bringing you three thrilling tracks in Embodied AI and Autonomous Driving, with a total prize pool of $100,000! Now get ready and join the competition!

Visit the challenge website: opendrivelab.com/challenge2025
And more on #CVPR2025: opendrivelab.com/cvpr2025

Remember the queue outside Room 442? Please mark the workshop in your CVPR registration to have enough space for all of you.

In CVPR 2025, our 3rd edition workshop will discuss the present and future of autonomous systems from a brand-new perspective. More on opendrivelab.com/cvpr2025/workshop/

🏟️ Okay, here we go with this year's challenges. In this 3rd edition of the Autonomous Grand Challenge, we have three tracks on Embodied AI and Autonomous Driving and a total cash pool of $100,000.

More information on opendrivelab.com/challenge2025/, and looking forward to your feedback!

🎇🎇 Wish you all a happy new year!