"wir müssen wissen; wir werden wissen!" - David Hilbert
orbiting near Sagittarius A*
Joined January 2020
- Tweets956
- Following67
- Followers131
- Likes60.9K
ConcurrentSquared retweeted
The METR report on RSI didn't get me to update strongly that we are really seeing a real doubling of AI lab research speed: maybe, maybe not. But that is not reassuring! The reason is that if you see a doubling you may be very close to the takeoff.
i should just write my thoughts up; for i have many thoughts, but no writings
but also i don't have that much time! also i apparently just have a ugh field around writing stuff
woaw!! an internal claude just solved a 20-year-old conjecture.
concurrent labs just released a proof of it. to do so, claude said 'calculemus!' to the calculus ratiocinator 12 times
the result proves that i'll never have a girlfriend
mit researchers said that this is 'a result'
ConcurrentSquared retweeted
🧵 Astra covertly underperforms on safety research tasks.
We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision to underperform was not verbalized in the CoT meaning our misalignment monitoring system did not detect this.
To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
tween concurrent thought that they would dress up like this at least once in the 2020s.
yet i haven't done so. clearly a market failure!!
as of nov. 2, 2026, 'weight-space programming' (via opus 5.5) doesn't beat adam + even a 3.4m parameter gpt on autocompleting basic scheme (much of it being solutions to sicp exercises)
fatebook.io/q/will-some-proc…
alternatively if i'm wrong and we don't die after gpt-7 solves interp. i will post a personal secret with the sha512 hash dcc01fec544c2eedf88324b57c697d3ab69d9182e7c32b9eb29eb9f62c9d351f0586e3ada0493b845b8171fc0d49c4deef5fd55cc267b090cba29ec241d74623
relatedly note that this task should be much easier than general language (for one: scheme has a type system, and a very restricted and formal grammar); i have the prediction fatebook.io/q/will-some-proc…, which should be easy if gpt-7 solves interp. via weight-based programming
ConcurrentSquared retweeted
meh, the more you dig into each experiment, the less magical it gets; it feels like a loose collection of cherry-picked demonstrations with a grand narrative built around them
Exp 1-2 are tiny (~20K params) with unusually easy to interpret architecture; Exp 1 is an MLP, where nearly all params are used in the first layer, so you can easily tell what kind of objects the network is trying identify
Exp 5 is about designing a game strategy (we already know that Astra is extremely good at this) and then converting it to a tiny (~20K params) MLP; entirely doable by a human, but of course would require a lot of effort
Exp 3 & 8 are closer to automated mechinterp using python and established techniques, rather than direct weight interpretation (at least according to chat)
to be clear, those are genuinely cool results, but I think people are reading too much into them
I don't think any of this should meaningfully change anyone's intuition about what models are capable of
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
i wish for happiness for everybody, free, and also gpt-4.5-base, if not gpt-6.2-base, and claude-fable-5.5-base
i can be trusted with the base text-completion models; no one will go away unsatisfied