Interested in retrieving (sub-)trajectories from enormous datasets like DROID? 🦣

Check out our retrieval code using DTW and vision foundation model features!

Code: github.com/WEIRDLabUW/S...
Website: weirdlabuw.github.io/strap

Happy holidays from UW Robotics!

Retrieving sub-trajectories boosts robustness of few-shot imitation learning by leveraging offline datasets!

w/ @jacob-berg.bsky.social, Bingqing Chen, @abhishekunique7.bsky.social, Jonathan Francis

Paper: arxiv.org/abs/2412.15182
Website: weirdlabuw.github.io/strap

Thanks to Bosch 🔧

STRAP retrieves semantically meaningful trajectories on large datasets like DROID making it a powerful general-purpose algorithm for robot trajectory retrieval!

🧵 5/6

STRAP uses features from vision foundation mode, e.g., DINOv2 as a metric space for the S-DTW matching.

This results in a non-parametric retrieval algorithm that can be used out-of-the-box and naturally scales to larger datasets.

🧵 4/6

Key insight: while entire trajectories contain multiple tasks and don’t match our demos, sub-trajectories are much more likely to be shared while still capturing temporal dynamics!

STRAP uses Subsequence Dynamic Time Warping (S-DTW) to find such relevant matches.

🧵 3/6

While behavior cloning memorizes the demos, STRAP learns a much wider distribution through its augmented dataset.

This leads to performance gains of 25% over prior retrieval methods (sim) and 40% over behavior cloning (real)!

🧵 2/6

Have some offline data lying around? Use it to robustify few-shot imitation learning! 🤖

STRAP 🎒 is a retrieval-based method that leverages semantic sub-trajectories in offline datasets to augment the training data.

🧵 1/6

I'm excited about scaling up robot learning! We’ve been scaling up data gen with RL in realistic sims generated from crowdsourced videos. Enables data collection far more cheaply than real world teleop. Importantly, data becomes *cheaper* with more environments and transfers to real robots! 🧵 (1/N)