hill climbing @PrimeIntellect
at the open source
Joined April 2022
- Tweets1.8K
- Following2.4K
- Followers1.9K
- Likes7.4K
Pinned Tweet
agi was the friends we made along the way
couldn’t agree more
Training models is the best job in the world. It's important to remember this is supposed to be fun. It's hard, confusing 90% of the time, and almost constantly surprising. Its always crazy to me to see people talk about AI research work sometimes in such nihilistic ways. You get to melt GPUs for a living, enjoy it!
It's important to remember negative results are just as valuable as positive results, finding a strong baseline is important even if it means you don't get to run as many experiments, and always be willing to change your mind when presented with evidence.
It feels like the pressure to have big splashing success all the time is dialed up to 11 with all of the funding and attention in the past couple of years. It might feel like focusing on the fundamentals instead of chasing novelty will hold you back, but strong science always wins out. The facts don't care about novelty.
If you work on a team remember to be kind to everyone and be a source of joy. Everyone else feels the pressure of this moment as well. It means more than you think to crack a joke, or encourage people when they get a bad result.
Daniel Auras retweeted
got tired of figuring out which RL library to use, which inference engine to pair it with, and how to make it all fit on my GPU-poor laptop🥲(RTX 4060 - 8gb vram).
so decided to put my codex subscription to some use and build something for myself!
(repo in the first reply)
Daniel Auras retweeted
Amazing BenchmarkGate [1] reveal by @xeophon where he found serious (even comical) problems in popular benchmarks:
AutomationBench: "[forces the models to] describe their request bodies in prose rather than in JSON Schema" 😅
He also fixed a number issues with the grader, which when combined with the above result in a near saturation of the benchmark!
OfficeQA Pro v2: the grader uses a regex and Python-based extraction, which results in rejecting many valid answers. Additionally, there is evidence of models using web access to retrieve the answer key from HuggingFace (receipts are in the post).
With these and other fixes, this again results in near saturation of the benchmark!
It's very impt to look at data and not trust metrics from third parties. Better yet, you should rely on your own private evals.
[1] BenchmarkGate: the gap between the [un]trustworthiness of benchmarks and the extent to which people rely on them.
don’t miss our latest piece by the benchlord himself @xeophon
huge!
now we can start hillclimbing this annoyance out of our models
Daniel Auras retweeted
Evals built by @PrimeIntellect residents, fully open:
- MazeBench by @patience_cave
- PMPP-Hard by @myainotez
- PretrainGym by @HuNtEr_J1324
- OpenTTD-Bench by @thesophiaxu
[and even more :) ]
Super happy to support such amazing work, check it out!
Daniel Auras retweeted
Tübi folks, catch @rasdani_ and @hallerite while they're here tomorrow!
Guaranteed to be fun and insightful
@hallerite and i are visiting tübingen tomorrow 🇩🇪
if you wanna chat multi-agent swarms, frontier RL and data or wanna nerd out about your own research shoot us a DM!
putting a 32b distill into the fineprint under “DeepSeek” is straight up criminal
you ABSOLUTELY need to take a look at the name of the model and how this benchmark table is constructed, this is absolute cinema
ty @mattsntlm
corollary:
co-design with hardware.
don’t get nerdsniped by analogies to biology.
first IPO then selling billions
🚀 0 → 120 BILLION tokens/day in just ONE month!
Just one month since onboarding Prime Inference to OpenRouter, we've scaled from zero to 120B tokens served daily. Note this is just a small portion of Prime Inference 🔥
openrouter.ai/provider/prime…
Daniel Auras retweeted
if you look at the replies and reposts of the main post AND the paid partnership posts most of them are fake and just bot farms at large scale
so against X ToS and quite easy to cluster since it's always the same accounts (there is a good blog post to make with "we asked grok 4.7 to identify all the clusters and ban the bot farms")
jev used this kind of slop agency and it definitely works and converted into real usage (and they raise a massive amount with this) so i can understand why people are doing it, it's just sad and should be fixed at the platform level quick otherwise everyone will do it and people will start to leave X
i find it really sad that X is becoming a pay to win platform and that even "good" releases are going through these fully slop/fake campaigns to get traction
i'm also pretty sure there is very shady stuff where they are boosting each other's posts with bot farms and more. would be great to investigate here @elonmusk and make sure the algorithm doesn't push their posts, you can very easily cluster that if you look at each release and the accounts amplifying 🙏
(not a dunk on recursive, recently a lot of people are doing that unfo and it's definitely working, posts often get over ~1M views)
Daniel Auras retweeted
Today we're releasing prime-rl v0.10.0 and verifiers v0.4.0
We added our new RL algorithm IPO as the default, tailored for async RL stability, as well as performance improvements and new model support
Daniel Auras retweeted
Moondream 3 Preview and Moondream 3.1 from @moondreamai are now on OpenRouter, served by @PrimeIntellect
Object detection and pointing work in plain chat: attach one image and start your message with /detect {"object":"coffee cup"} or /point {"object":"croissant"}
Daniel Auras retweeted
Prime Intellect is buying Moondream, a startup with fewer than 10 employees, for its expertise in speeding up AI inference.
The acquisition could help Prime Intellect lower computing costs and attract customers building robotics and AI agent applications.
Full story: thein.fo/4smew7f
