PI of the CLVR lab. Associate Professor @KAIST_ai. Prev: assist. prof. @USC, postdoc @Stanford, CS PhD @MIT, BA @UCBerkeley. AI, Robotics, Vision, ML.
Seoul, Republic of Korea
Joined July 2017
- Tweets62
- Following437
- Followers1.4K
- Likes227
Joseph Lim retweeted
Very excited to start sharing some of the work we have been doing at Amazon FAR. In this work we present OmniRetarget, which can generate high-quality interaction-preserving data from human motions for learning complex humanoid skills.
High-quality re-targeting really helps the reinforcement learning.
Why? The control policy optimization landscape is much nicer for (near)feasible trajectories than for trajectories with artifacts like (e.g.) foot skating or ground penetration.
Humanoid motion tracking performance is greatly determined by retargeting quality!
Introducing 𝗢𝗺𝗻𝗶𝗥𝗲𝘁𝗮𝗿𝗴𝗲𝘁🎯, generating high-quality interaction-preserving data from human motions for learning complex humanoid skills with 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 RL:
- 5 rewards,
- 4 DR terms,
- Proprio. ONLY,
- NO history/curriculum.
Ready for agile, human-like 🤖? (Best with 🎧)
🔗 omniretarget.github.io 🎥
1/9
Joseph Lim retweeted
It's almost time for #CoRL 2025! A reminder that we're hosting the Data in Robotics workshop this Saturday Sept 27th. We have a packed schedule and are also attempting to livestream the event for those who can't attend in person.
Joseph Lim retweeted
We have our speakers: @JosephLim_AI @KarlPertsch @mayacakmak , Serena Booth, @JasonMa2020 @shahdhruv_ @MashaItkina
We also have a great panel lined up from 3-3:40pm!
@svlevine @yukez @MashaItkina @SudeepDasari @ashwinb96 @JasonMa2020 @shahdhruv_
Joseph Lim retweeted
Honored that our @RL_Conference paper won the Outstanding Paper Award on Empirical Reinforcement Learning Research!
📜Mitigating Suboptimality of Deterministic Policy Gradients in Complex Q-Functions
📎openreview.net/forum?id=H3jc…
Grateful to my advisors @JosephLim_AI and @ebiyik_!
At @RL_Conference🍁, I'm presenting a talk and a poster on Aug 6, Track 1: Reinforcement Learning Algorithms.
We find that Deterministic Policy Gradient methods like TD3 often get stuck at local optima under complex Q-functions, and propose a novel actor architecture!
🧵
Joseph Lim retweeted
I've been historically bearish on skill-based RL and @Jesse_Y_Zhang consistently shows me I'm probably wrong. Come check out EXTRACT at 1600 at @corl_conf (poster #11) to see him yank semantically meaningful skills out offline data by leveraging PTLMs.
arxiv.org/abs/2406.17768
Joseph Lim retweeted
This work was guided by Jesse's co-advisor's @ebiyik_ and @JosephLim_AI, but I think we all agree that Jesse is effectively running his own little lab. If you're thinking about hiring a postdoc in the area of RL for robotics, Jesse comes a ton of insight and mentoring experience!
Joseph Lim retweeted
Having humans annotate data to pre-train robots is expensive and time-consuming!
Introducing SPRINT:
A pre-training approach using LLMs and offline RL to equip robots w/ many language-annotated skills while minimizing human annotation effort!
URL: clvrai.github.io/sprint/
🧵👇
Joseph Lim retweeted
Excited to present PATO: Policy Assisted TeleOperation, our recent work on scaling robot data collection!
PATO uses a policy trained on prior data to assist the user during data collection, making teleop easier and even allows to teleop multiple robots simultaneously.
🧵👇
Joseph Lim retweeted
Excited to share our work on solving IKEA furniture assembly by chaining a long sequence of skills at #CoRL2021!
📜 openreview.net/forum?id=K5-J…
🔗 clvrai.com/skill-chaining
Come visit our poster at Tues 5pm (9am PST)!
Joint work with @JosephLim_AI , @AnimaAnandkumar , @yukez
Joseph Lim retweeted
New paper on *Skill-based Learning with Demonstrations (SkiLD)*!
While current imitation learning follows the _low-level actions_ in the demos, SkiLD follows the demonstrated _skills_.
SkiLD enables efficient demo-guided RL & imitation learning on long-horizon tasks!
1/N
Joseph Lim retweeted
What if robots could be as easy to use as computers are today? Introducing Intrinsic: a new Alphabet company creating software to make industrial robots easier for businesses, entrepreneurs, and developers to use. blog.x.company/introducing-i…
Joseph Lim retweeted
I recommend “you and your research” to anyone starting out in research! You don’t need to carry everything forward with you, but it is a fun read that made me think the nature of truly important research contributions (+ avoiding working on questions that dont ultimately matter).
Finally reading "You and Your Research" by Richard Hamming and it's so good (!!)
nicely describes the maneuvers that you rapidly pick up by dealing with bureaucracies and hierarchies
cs.virginia.edu/~robins/YouA…
Joseph Lim retweeted
I have been chasing to discover the emerged knowledge in DNN for a long time. We have a recent work showing that different levels of semantics emerge in deep generative representations when trained for scene synthesis.
Paper & Webpage: ceyuan.me/SemanticHierarchyE…
Joseph Lim retweeted
If all training images for a GAN/VAE/PixelCNN have 2 objects, will they only generate images with 2 objects? If trained on (🔵,💙,🔴), will they also generate ❤️? Find out in @shengjia_zhao's blog post on generalization and bias for generative models.
👉ermongroup.github.io/blog/bi…
Joseph Lim retweeted
The website for the #NeurIPS2019 Deep RL Workshop is now live!
Paper submissions due September 9th.
Co-organized with @jachiam0, @chelseabfinn
Joelle Pineau, David Silver, Satinder Singh, Carlos Florensa, Chris Grimm, @hrtang_alex, @vivek_veeriah
sites.google.com/view/deep-r…
Joseph Lim retweeted
Excited to share our NeurIPS workshop: Learning Transferable Skills. skillsworkshop.ai/
On how to learn AND test for task-independent skills. Also, looking for perspectives/methodologies from other fields that might be overlooked in the AI community!
Papers: Sep 9th.
Happy to share that I, along with @danny_lange, @marwanmattar, @MaCroPhilosophy, and @BenBeyret are organizing a NeurIPS2019 workshop on "Learning Transferable Skills!" skillsworkshop.ai/