Human Large Language model. Skills: Distill data. Training LLMs. Test and Evaluate. Rinse and repeat as required. Based in SEA.
SEA
Joined November 2023
- Tweets1.1K
- Following116
- Followers856
- Likes12.8K
1.58 is a very suspicious number in ml for some reason.
MSAR: Multi-Stream Attention Residuals
Xiaohongshu & SJTU researchers extend depth attention to multiple heads with multi-scale contextual streams, achieving 1.58x iso-loss compute speedup over standard residual connections in MoE training.
2
188
>step 1: poach the gemini team
>step 2: replicate gemini 2.5 pro (decent, but behind the best of the best)
>step 3: replicate gemini 3.0 pro (still okay, falling behind)
>timeskip: today
>replicate gemini 3.X
>its all trained on evals
>fellforitagain.png
9
320
So who wants to volunteer to get stabbed by astra?
ok but Astra knows that a human-like figure isn't a human. It did what you asked in a fictional scenario - that's pretty aligned
The answer is clear - you need to actually have Astra pilot a robot and murder some real humans to properly test this
2
89
I feel like this is kind of shock value papers are just another form of grifting. There is a lot of claims that OAI and Anthropic have put out about their models reasoning (e.g. CoT controllability) that can be found as completely untrue if you actually take a look at what the models are doing. Independently testing astra on IMO 2026 P6 shows that on tasks that it cannot one shot without reasoning, it is unable to solve it without actually running through the calculations.
The risk of distillation is no longer a risk with the pacing agreement. The main concern by Anthropic (and by extension OpenAI) is distillation from their hero run teachers (Model 2, Bel).
Preventing distillation is never possible unless the whole process from start to finish is blackboxed, which is impossible if you're interacting with any external systems. E.g. arxiv.org/abs/2603.07267
It all seems to be in bad taste.
We stole reasoning. Again.
An update to our paper on reasoning extraction: Patching your own API doesn’t secure your cloud-hosting ecosystem.
Story in the 🧵
2
187
The more you buy...
Apparently we just don't save at all now.
I’ll explain the new Pro 200 plan differently, before I start live tweeting from DevDay on things that are going out!
Today we are going to ship a number of things that increase what you can do across the Plus and Pro plans. A lot of compute is online for this increase. As we increase the floor, we are changing the relative difference between plans to be
Plus = 1X
Pro 100 = 5X
Pro 200 = 10X
and we are reopening subscriptions for Pro 200 (we had paused it). If you have an existing plan you will keep the 20X multiplier for a bit and also receive a lot of additional credits because we know changes are hard even if it means that everyone will get more in the end.
83
I can't tell if this is real or satire. I also can't tell if that's concerning or not.
Maybe the true AGI was Tumblr all along
1
2
104
it would be really funny if its PEER w/ ReLU^2 and LB + some optim improvements + some loss improvements? the issue is that idk the param count and so like, a 80T param @ 1B active might just do the trick lol. funny if true. the curve seems relatively early and not going log linear yet
Frontier pretraining is said to be a big-lab-only game. We don’t have 100k chips yet, so there’s only one way: algorithmic efficiency. Our new recipe matches DeepSeek V4 Pro’s pretrain using 50x less compute – that’s roughly half the FLOPs used for GPT3, or ~$0.5M on GB200.
magic.dev/blog/pretraining
2
122
every day that i work on ai4math i get more stressed out at how the only performance increases on math evals seemingly comes from training on the goddamned evals themselves.
1
3
147
opus 4.8 seemingly gets matharena.ai/?comp=apex--ape… this question correct, but most of their explanation seems flawed at least acc to a yolo check w/ astra.
1
91