ML @Woven_ToyotaJP; Senior Research Scientist @ToyotaResearch. Surf @ Santa Cruz and Ichinomiya. PhD @TTIC_Connect, @UChicago alum.
東京都、日本
Joined August 2020
- Tweets57
- Following549
- Followers273
- Likes487
Igor Vasiljevic retweeted
Releasing VLA Foundry: an open-source framework that unifies LLM, VLM, and VLA training in a single codebase. End-to-end control from language pretraining to action-expert fine-tuning — no more stitching together incompatible repos.
Igor Vasiljevic retweeted
My team is looking for highly motivated research interns this summer with strong backgrounds in 3D representations for robotics and scene understanding. If you’re interested, please feel free to DM me!
jobs.lever.co/tri/95fba28c-9…
Igor Vasiljevic retweeted
🚀Thrilled to share what we’ve been building at TRI over the past several months: our first Large Behavior Models (LBMs) are here! I’m proud to have been a core contributor to the multi-task policy learning and post-training efforts.
At TRI, we’ve been researching how LBMs can help robots learn faster, better, and more efficiently.
The key takeaways:
✅ We built an evaluation pipeline to benchmark LBM performance with real 𝐬𝐭𝐚𝐭𝐢𝐬𝐭𝐢𝐜𝐚𝐥 𝐜𝐨𝐧𝐟𝐢𝐝𝐞𝐧𝐜𝐞
✅ Pre-training on hundreds of tasks makes models more robust—plus, we can teach new, complex tasks with 80% 𝐥𝐞𝐬𝐬 𝐝𝐚𝐭𝐚
✅ The bigger and more diverse the pre-training, the better the results
Check out our overview video, webpage and paper for more details:
✨youtube.com/watch?v=DeLpnTgz…
🌎 toyotaresearchinstitute.gith…
📄 arxiv.org/pdf/2507.05331
We hope this work helps move the field of robotics forward!
TRI's latest Large Behavior Model (LBM) paper landed on arxiv last night! Check out our project website: toyotaresearchinstitute.gith…
One of our main goals for this paper was to put out a very careful and thorough study on the topic to help people understand the state of the technology, and to share a lot of details for how we're achieving it.
youtube.com/watch?v=BEXFnru5…
Igor Vasiljevic retweeted
Excited to share our new work on multi-object scene completion and grasp pose estimation from a single RGB-D image!
Kudos to @s1wase and the incredible team from @ToyotaResearch, @WbyT_Tech, and @CarnegieMellon.
Come chat with us at #CVPR2025 to learn more.
#CVPR2025 starts in two days, and can’t wait to share our new work! 🎉 We present ZeroGrasp, a unified framework for 3D reconstruction and grasp prediction that generalizes to unseen objects.
Paper📄: arxiv.org/abs/2504.10857
Webpage🌐:sh8.io/#/zerograsp
(1/4 🧵)
Igor Vasiljevic retweeted
Huge shoutout to the whole OpenThoughts team, especially @etash_guha @ryan_marten @NeginRaoof_ @GeorgeSmyrnis1 @AlexGDimakis @lschmidt3
Finding a great team is a huge factor in the success of any project, and it's been an absolute pleasure working with this team!
Thank you to the whole OpenThoughts team for yet another great effort!
@etash_guha, @ryan_marten, @sedrickkeh2, @NeginRaoof_, @GeorgeSmyrnis1, @hbXNov, @marnezhurina, @MercatJean, @trungthvu, @ZayneSprague, @suvarna_ashima, @FeuerBenjamin, @cliangyu_, @codezakh, @esfrankel, @sachingrover, @carolineschoi, @Muennighoff, @shiye_su, @WanjiaZhao1203, @jyangballin, @sayshrey, @kartiks26387917, @charlie_jcj02, @YCEthanDeng, @sarahmhpratt, @RamanujanVivek, @JonSaadFalcon, @jeffwpli, @achalddave, @AlbalakAlon, @karora4u, @wulfebw, @chegday, @gregd_nlp, @sewoong79, @mohitban47, @GabrielSaadia, @adityagrover_, @kaiwei_chang, @Vaishaal, @SkyLi0n, @Mike_A_Merrill, @tatsu_hashimoto, @YejinChoinka, @JJitsev, @HeckelReinhard, @madiator, @AlexGDimakis, @lschmidt3
(N/N)
Igor Vasiljevic retweeted
FastMap: Revisiting Dense and Scalable Structure from Motion
Jiahao Li, @__whc__, @mzubairirshad, @vslevic, Matthew R. Walter, Vitor Campagnolo Guizilini, @gregshakh
tl;dr: replace BA with epipolar error+IRLS; fully PyTorch implementation
arxiv.org/abs/2505.04612
Igor Vasiljevic retweeted
Introducing ✨Posed DROID✨, results of our efforts at automatic post-hoc calibration of a large-scale robotics manipulation dataset.
We provide:
🤖 ~36k calibrated episodes with good quality extrinsic calibration
🦾 ~24k calibrated multi-view episodes with good-quality multi-view camera calibration
✅ Quality assessment metrics for all provided camera poses
To achieve this, we utilize:
1️⃣ Auto Segment Anything (SAM) based filtering (Camera-to-Base Calibration)
2️⃣ Tuned CtRNet-X for bringing in additional cams (Camera-to-Base Calibration)
3️⃣ Pretrained DUST3R with depth-based pose optimization (Camera-to-Camera Calibration)
Try it out at: droid-dataset.github.io/
Learn more at:
🌐 arXiv: arxiv.org/pdf/2403.12945
📄 Blog: medium.com/p/4ddfc45361d3
🧵 1/n
Igor Vasiljevic retweeted
1/
DeepSeek-VL is trained from DeepSeek LLM
Qwen-VL is trained from Qwen-7B
PaliGemma is trained from Gemma-2B
Is this really the best way to train a VLM?
What if we had access to model checkpoints -- would it be better to train with images before the LLM fully converges?
🧵
Igor Vasiljevic retweeted
thrilled about this awesome initiative! super excited about reasoning models and building these datasets 🚀
Igor Vasiljevic retweeted
We're seeing more and more that small models trained on high-quality datasets can perform very well.
Together with our collaborators at DCLM, we trained strong 1B models and openly release everything!
Check it out at huggingface.co/TRI-ML/DCLM-1…
Igor Vasiljevic retweeted
Check out our latest models here:
huggingface.co/TRI-ML/DCLM-1… for the base model, and
huggingface.co/TRI-ML/DCLM-1… for the IT model with evaluations comparing to other strong small models
Igor Vasiljevic retweeted
As always, this wouldn't be possible without all the DataComp-LM collaborators and a special thanks to @ToyotaResearch, Apple, and UW!
Igor Vasiljevic retweeted
Excited to share our new-and-improved 1B models trained with DataComp-LM!
- 1.4B model trained on 4.3T tokens
- 5-shot MMLU 47.5 (base model) => 51.4 (w/ instruction tuning)
- Fully open models: public code, weights, dataset!
Igor Vasiljevic retweeted
We've publicly released our DataComp-LM models: Truly open 1B and 7B models that's competitive with state-of-the-art (llama3, qwen2, gemma, ...) on most benchmarks, but with a public training recipe, dataset, and code! (1/3)
Igor Vasiljevic retweeted
- tons of new cool work on large rnns (@RWKV_AI, mamba2 @tri_dao @_albertgu, just read twice @simran_s_arora, etc)!
- but pretraining is expensive
- our recipe for linearizing llms into rnns was accepted to @COLM_conf! #COLM2024
- we train SOTA rnns & show limitations of rnns
Igor Vasiljevic retweeted
I am really excited to introduce DataComp for Language Models (DCLM), our new testbed for controlled dataset experiments aimed at improving language models. 1/x
Igor Vasiljevic retweeted
Check out DataComp for language models! Open data, open code, open training recipe, and close to Llama3-8B performance. This has been a labor of love over the last year, a huge thanks to all the collaborators for helping make this happen!
Igor Vasiljevic retweeted
Self-Supervised Geometry-Guided Initialization for Robust Monocular Visual Odometry
Takayuki Kanai, @vslevic, @vitorguizilini, Kazuhiro Shintani
tl;dr: zero-shot monocular depth estimation->geometric prior->initialize dense bundle adjustment
arxiv.org/pdf/2406.00929
Igor Vasiljevic retweeted
Starting the #RoboNeRF workshop at #icra2024 with our first speaker @leto__jean. Jeanette's talk is on Grasping with NeRF! Come check it out at Conference Center 419!
Igor Vasiljevic retweeted
Our #Robotics team and @ToyotaResearch are working together to research a category-level 6D pose estimation to manipulate a large variety of household objects.
Watch how we found a way to improve the performance with Diffusion Models👇
woven-planet.github.io/Diffu…