THE VERSION NUMBER DECIDES HOW CAREFUL YOUR AGENT IS
microsoft shipped the same breaking change as a patch, a minor and a major:
> 13% of patch-labeled updates merged unchecked
> 4% when labeled major
> opus 5.5 never skipped the check
the cheaper the model, the more it trusts the label
so the real cost of a small model isn't tokens, it's the checks it skips
AI agents picked up one of our worst habits.
A guy from Microsoft gave coding agents the same library update about 1000 times. The code was identical every time, and it had a change inside that breaks things. The only difference was the label.
When the update was called a small patch, Sonnet checked what changed in just 2 runs out of 10. When the exact same code was called a major release, it checked 10 out of 10.
That means the agent doesn't really read what changed. It looks at the version number and decides how careful to be.
Most of us did the same thing. We saw a small patch, hit update and moved on. Agents learned from our code, so they learned that too.
The fix is one line in your agent's instructions. "Before any dependency update, read the changelog and run the tests, no matter how small the version bump looks."
That way the agent checks every update, not just the ones that look major.
1
1
24
source: microsoft devblog, "do ai coding agents care about semver?", nov 9
developer.microsoft.com/blog…
7
67% IN ONE PROMPT FOR YOU, 46% FOR ME
same three.js prompt, fable 5.2 vs sonnet 5.5, a lighthouse in a night storm
> fable: 6:36, 46% of my 5-hour limit
> sonnet: 11:23, 7% of my 5-hour limit
> fable also took 13% of my weekly fable limit
that's about 7 fable scenes a week on pro
fast is expensive. pick the model per task
Fable 5.2 drained 67% of my 5 hr limit in a single prompt 💀
My honest review (not paid, good and bad)
The good:
It handles most work, it's insanely good at finding and fixing bugs, and the design taste is way better than Opus. It's also really good at 3D modeling, websites and data analysis
The bad:
Very expensive and token hungry. The jump over Opus 5.5 isn't really worth the price
Would I pay for it after Nov 16?
No. I'll rather stick with Opus 5.5
What did you think of it?
1
1
46
how i measured: same prompt, new chat, same effort, stopwatch. limit % read from settings before and after each run
format borrowed from @EnvolDev's ice cathedral test
last round: xitter.gq/Spinzu/status/21201771…
ROUND 4: FABLE 5.2 VS SONNET 5.5. 1:51 VS 1:52
same checkers prompt, forced jumps on
both passed every check: forced jumps, multi-jumps, kings, 10+ clean moves
sonnet added draw rules and undo on its own
fable is the new model at $8/$40 per 1M tokens. sonnet is 3-0
one second apart. who takes it?
24
ROUND 4: FABLE 5.2 VS SONNET 5.5. 1:51 VS 1:52
same checkers prompt, forced jumps on
both passed every check: forced jumps, multi-jumps, kings, 10+ clean moves
sonnet added draw rules and undo on its own
fable is the new model at $8/$40 per 1M tokens. sonnet is 3-0
one second apart. who takes it?
play both yourself, try skipping a jump:
fable 5.2: claude.ai/artifact/FWsMRoBS6…
sonnet 5.5: claude.ai/artifact/ETidPgpVt…
24
rules: same prompt, same effort, one stopwatch. replies decide this round, 24h
@jBOOGIEai called it: "a missed jump quietly breaks the whole game"
@k0dzilla: "a mini game breaks the second the rules do not hold across turns"
@EnvolDev turn-based round is out, as promised
round 3: xitter.gq/Spinzu/status/21183767…
ROUND 3: SONNET 5.5 VS GPT-6 LUNA MAX. SAME PROMPT, NOW WITH A STOPWATCH.
sonnet took round 2 on the poll, but the replies on sonnet vs argon split:
@k0dzilla: speed wins if the output is usable
@has_anver, on argon: stays locked to the beat even on the later loops. sonnet's faster but next to it it feels flat
so round 3 checks both against a new opponent:
> same 8-bit arcade prompt, one shot
> stopwatch from send to finished code
> first play, then 3 replays
sonnet 5.5: code in 4m 47s. the ship flies over each letter, full title by 5s, same on all 3 replays
gpt-6 luna max: code in 7m 07s. letters drop one by one, full title in under 3s, same on all 3 replays
sonnet writes faster. luna's intro plays faster
speed or showmanship? top or bottom?
1
2
52
67 DAYS SINCE OPENAI'S LAST FRONTIER MODEL
one more week without 6.2 Astra and it's the longest wait since GPT-5 to 5.1
i'm keeping a seat open in my model tournament
sonnet 5.5 is 3-0 so far
Astra gets the same prompt the day it ships
rule stays the same: one prompt, one stopwatch, you vote TOP or BOTTOM
It has been 67 days, or just over 2 months, since OpenAI last released a frontier model
If 6.2 Astra fails to release this week, it will be the longest gap between OpenAI frontier model releases since GPT-5 -> 5.1
❄️
1
153
Around 1940 Les Paul bolted strings and pickups onto a 4x4 block of pine. No hollow body. Just a string, a magnet and a coil.
I gave Claude the same starting point: no 3D models, no audio samples, only the physics.
> PHYSICS - string tension, fret math and pickup positions written down first
> BUILD - every part of the guitar generated in code from those numbers
> CHECK - each string measured live from the sound you hear. 6/6 within 0.1 cent
6 strings. 22 frets. fret 12 measures 2x the open string on all six, off by 0.0001 at most.
every note is a vibrating string in code, not a recording.
1
1
56
ROUND 3: SONNET 5.5 VS GPT-6 LUNA MAX. SAME PROMPT, NOW WITH A STOPWATCH.
sonnet took round 2 on the poll, but the replies on sonnet vs argon split:
@k0dzilla: speed wins if the output is usable
@has_anver, on argon: stays locked to the beat even on the later loops. sonnet's faster but next to it it feels flat
so round 3 checks both against a new opponent:
> same 8-bit arcade prompt, one shot
> stopwatch from send to finished code
> first play, then 3 replays
sonnet 5.5: code in 4m 47s. the ship flies over each letter, full title by 5s, same on all 3 replays
gpt-6 luna max: code in 7m 07s. letters drop one by one, full title in under 3s, same on all 3 replays
sonnet writes faster. luna's intro plays faster
speed or showmanship? top or bottom?
round 3 goes to sonnet. the one clear pick came from @jBOOGIEai, and by @k0dzilla's rule it fits too: usable, consistent, code 2 min faster. luna still played the faster intro. round 4 gets a new prompt
1
2
40
$0.18 FOR A CODE REVIEW THAT CAUGHT 2 ISSUES THE $0.37 ONE MISSED.
opened her full bake-off. 4 codex runs, same prompt, same branch:
> sol high: $0.369, 93s, reproduced the bugs, ran the tests
> 2x terra: $0.18, 17s, static only, 1 wrong claim
> output was under 1% of sol's 528K tokens. you pay for reading, not writing
so terra finds, sol proves. cheap pair first, sol only on what they flag.
I stopped using GPT-6.1 Sol for code review. GPT-6.1 Terra is cheaper and it beat Sol.
Same PR, same prompt:
🐢 GPT-6.1 Sol (high): 93 seconds, $0.37
⚡ 2× GPT-6.1 Terra (high): 17 seconds, $0.18
The cheap pair was 5× faster, cost half as much, and caught 2 bugs Sol missed. Sol's only edge was running the tests to prove its bugs.
Since every model misses bugs, even Sol, I never rely on one reviewer. My setup now:
1. Two Terra subagents review the PR in parallel
2. A third Terra has them debate and settle on the final findings
2
2
81
her full bake-off with all 4 runs, token splits and the one wrong claim: share.onorca.dev/a/VIruT485I…
28
round 2 for context: xitter.gq/Spinzu/status/21179990…
ROUND 2: SONNET 5.5 VS GEMINI 4 ARGON. SAME PROMPT, ONE SHOT.
sonnet was done in 3.6 seconds. argon took 6, but synced every letter to its own chiptune
speed or showmanship? poll below ↓
1
14
ROUND 2: SONNET 5.5 VS GEMINI 4 ARGON. SAME PROMPT, ONE SHOT.
sonnet was done in 3.6 seconds. argon took 6, but synced every letter to its own chiptune
speed or showmanship? poll below ↓
CLAUDE SONNET 5.5 VS GEMINI 3.8 FLASH. SAME PROMPT, ONE SHOT.
kinder ran a four-model showdown on one prompt. i ran a smaller fight: one claude, one gemini, the cheaper tiers.
the prompt: an 8-bit arcade intro. a ship flies in, "SPINZU" appears letter by letter, stars scroll, chiptune plays. one html file.
top: claude-sonnet-5-5-high
bottom: gemini-3.8-flash-high
> sonnet: letters explode on impact, screen flash, the ship parks under the title
> flash: a whole arcade cabinet with crt glass and "push start button"
> both wrote their own music in web audio. no files, no samples
vote in the replies: top or bottom?
the winner fights a much bigger model in round 2. follow so you don't miss it
round 2 goes to sonnet on the poll. the replies split the other way, @has_anver and @jBOOGIEai went showmanship. so round 3 keeps both questions: sonnet vs gpt-6 luna max, stopwatch plus 3 replays
1
21
CLAUDE SONNET 5.5 VS GEMINI 3.8 FLASH. SAME PROMPT, ONE SHOT.
kinder ran a four-model showdown on one prompt. i ran a smaller fight: one claude, one gemini, the cheaper tiers.
the prompt: an 8-bit arcade intro. a ship flies in, "SPINZU" appears letter by letter, stars scroll, chiptune plays. one html file.
top: claude-sonnet-5-5-high
bottom: gemini-3.8-flash-high
> sonnet: letters explode on impact, screen flash, the ship parks under the title
> flash: a whole arcade cabinet with crt glass and "push start button"
> both wrote their own music in web audio. no files, no samples
vote in the replies: top or bottom?
the winner fights a much bigger model in round 2. follow so you don't miss it
this is f*cking adorable. four models, one prompt: an octopus making breakfast with all eight arms.
Gemini 4 Carbon · Gemini 4 Argon · Claude Opus 5.5 · GPT-6 Astra
eight arms, eight jobs - flipping pancakes, pouring coffee, reading the recipe, frying eggs, all at once. and honestly? they all pulled it off. different vibes, different kitchens, same impossible multitask nailed.
I didn’t touch a thing and I’m not scoring them. so it comes down to taste: whose kitchen would you actually eat breakfast in?
Carbon, Argon, Opus or Astra? drop your pick