Adam Elkassas

This

When designing RL tasks you realize that half of it is cybersecurity engineering

I never once was able to make meaningful progress with the best model (opus with CC) without human in the loop. Never has an improvement loop on our codebase actually worked in practice without me babysitting it. I think this says a lot about where recursive self-improvement is, if human's still play a big part here. I'm not talking about a long horizon task. I'm talking about an improvement loop where a rough goal is defined but not the specifics.

Let the data talk, leave your elitist ego please

Let the data quality talk, don't take our word for it

It's interesting how we're building RL datasets to create lift on benchmarks such as TB4, but the datasets we create are cleaner and harder to reward hack than TB4 itself.

Agreed. Especially for coding tasks where it’s all or none rewards it makes it incredibly difficult to create tasks that are just hard enough for the model to pick up new capabilities off of huge number of rollouts

Acquiring raw data is easy, building tasks is harder, delivering clean datasets is even harder

Maybe the data companies that claim ARR by multiplying their 1 day revenue by 365 are just smarter than you

X creates a collective short attention span in tech fueled by never-ending FOMO

Terra 6.1 is Terra-ble

If you use the word AI native to describe your company then you’re more than likely obsolete

Training models is easy. Creating curated datasets is not.

Hot take: the ramps, coinbases, etc doing their own harness and model work is just an overblown trend that will end up being a waste of money because a highly specialized startup is just going to be providing better solutions out the box. It's way harder than you think to train your own models and create RL tasks out of raw data

The same people bitching about benchmarks not being perfect also don't understand how difficult it is to build tasks in the first place vs. how easy it is to find defects. Cool you fired up Claude code and found some random defect in a task - then why don't you build a better benchmark bro? It's because you can't.

With all these "neolabs" trying to do everything from inference to sandboxing to RL to data acquisition, maybe all you need is some semblance of focus to win

A lot of running in place in the AI market

Idt blind allocation of compute is going to be a viable strategy in the coming years

Great writeup by my co founder @ArjunRajJain on @predotdev’s shared memory layer, unified across CLI and web with its native graph search