Pinned Tweet
We've been quiet for 2 months building what we think will be the future of physical data.
We're back now, welcome Interlatent Dataroom.
Pasted the Huggingface link to my agent to import it on Interlatent and told it to summarize it, here's what we got.
it also found that 156 episodes are unannotated with 0 subtasks
In collaboration with @HuggingFace, we’ve released the largest open robot dataset, alongside their first open System 1-System 2 model that can reason over step-by-step actions.
FineART is built with 40K+ demonstrations across 151 tasks and 500K subtask labels.
Together, the dataset and model were designed for each other, and both are available on @LeRobotHF where robotics developers work today.
@LeRobotHF @jadechoghari if this was a mistake u could maybe use us to check next time
here's all the unlabeled episodes:
1277, 2879, 3231, 3239, 3393, 3634, 3857, 3870, 4993, 5650, 6636, 6903
7729, 7762, 7864, 7865, 8397, 9509, 10612, 10781, 11003, 11298, 11657, 11921
11974, 13899, 14307, 14426, 15277, 15571, 15897, 15980, 16059, 16278, 16628, 18287
19157, 19281, 19762, 19850, 19996, 20066, 20251, 20357, 20374, 20388, 20535, 20654
20761, 20929, 20982, 21139, 21266, 21342, 21346, 21565, 21684, 22045, 22104, 22938
23013, 23167, 23269, 23873, 23938, 25328, 25333, 25340, 25341, 25373, 25394, 25440
26171, 27129, 27259, 27269, 27517, 27994, 28044, 28177, 29431, 30018, 30026, 30044
30067, 30072, 30074, 30110, 30225, 30237, 30246, 30254, 30304, 30332, 31100, 31923
32015, 32310, 32314, 32338, 32350, 32369, 32386, 32471, 32650, 32779, 32811, 32818
32899, 33327, 33328, 33329, 33371, 33433, 33471, 34602, 34635, 35113, 35382, 35385
35874, 36646, 36699, 36737, 36771, 36908, 36931, 36939, 36951, 37129, 37389, 37860
38007, 38059, 38071, 38080, 38198, 38331, 38755, 38822, 38823, 38833, 38853, 39044
40004, 40237, 40239, 40242, 40245, 40246, 40247, 40275, 40407, 40408, 40416, 40469
how long til openai releases GPT-7-Robot and it's a general purpose model post-trained heavily on robotics that runs as fast as VLAs but solves 95% of tasks
realistically what is the moat for these VLA companies once that happens
they don't have frontier lab clusters or distribution
Just pasted Codex outputs into Claude and vice-versa back and forth a couple times
felt something die inside me
Quick writeup on how we use agents to process robotics data (ego, dual-arm)
interlatent.com/blog/dataroo…
All 1000 episodes densely annotated by our agent in 10 mins!
attached is my workflow - i just sent the link to HF to import, then told it to annotate all the episodes in batch
view dataset here:
interlatent.com/dataroom/979…
"this will never be on the road how long will this take to ship elon is just doing this for optics"
so what? it's a flying fucking car
when did we stop finding cool thing cool
civilizational announcement
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Annotating and processing 15k episodes (5.7+ days, 200+ tasks) of open-source robot eval datasets containing failures
For safe deployment, we must cheaply understand failures and unexpected behavior
It's so easy to create reward pipelines with Astra on Interlatent
This one's a cup stacking where we increase the reward on each cup stacked, the timeline's are completely accurate to the millisecond
it took 1 prompt
i see some frame drops in this dataset as well, going to create a pipeline rn to detect these as well