RoboChallenge approved my account and I am working on submitting SmolVLA/base and SmolVLA-fl models for evaluation.
In other news, the RoboChallenge team confirmed plan to add @LeRobotHF so100/so101 arms to the remote test lab for benchmarks.
They are "excited to enable more DIY enthusiasts to participate."
(1/N)🚨How well does embodied intelligence perform in the real physical world? We are thrilled to introduce RoboChallenge co-initiated by @Dexmal_AI and @huggingface : the first large scale real-robot-based benchmark platform of embodied intelligence.
How far has embodied AI gone
Check out this first real world VLA manipulation evaluation, ran by tirelessly ARX, Franka, UR5 and Aloha arms
Table 30 are trivial for human but still very difficult for robots. pi0.5 is leading but still scores <50%
A long way to go! Link below: