Grateful for the fantastic collaboration with @ashwinb96 @oier_mees @SurajNair_1 @seohong_park @wulfebw @MashaItkina @ben_eysenbach @svlevine @tkollar @Ben_Burchfiel
Paper: arxiv.org/abs/2410.20018
Website: ghil-glue.github.io/
2
233
We study the interplay between image/video foundation models and low-level robot control policies, and present GHIL-Glue, a simple method for aligning image/video foundation models and low-level policies for robotic control.
1
1
153
But generative models may produce photo-realistic yet physically infeasible goal images that confuse low-level policies. They can also generate images with subtle yet harmful visual artifacts. How does this affect the performance of the robot agent?
1
117
Image and video foundation models can transfer knowledge from Internet data to robot agents by generating goal images that show low-level control policies how to complete a diverse range of tasks.
1
176
Image/video models can transfer knowledge from Internet data to robot agents by generating goal images. But what happens when images have harmful visual artifacts? We present GHIL-Glue, a method to align image/video models and low-level policies.
ghil-glue.github.io/