Developing Dosu https://xitter.cf/t.co/fyEGY7J6p3

Joined August 2021
Anthropic released Claude Haiku 5.5 today, so we ran the 5.5 family through our coding agent token usage benchmark. Haiku, Sonnet, and Opus each did 5 tasks, 3 times each, in Claude Code at its default medium effort. Haiku 5.5 spends a significant amount of time on reasoning and planning, which is typical for how smaller models increase intelligence.
7
3
8
446
30 agents minutes or 2-2.5 engineering weeks
1
3
273
early results on Luna vs Jev vs Decisions API on a few internal classification tasks tldr; they are very comparable on performance and speed
2
1
3
387
We ran the new models through our coding agent token usage benchmark. @OpenAI's GPT-6.1 Sol vs @AnthropicAI's Claude Sonnet 5.5 and Claude Opus 5.5. For every one of them, the biggest line item was context. Claude still does more formal planning than GPT. GPT-6.1 Sol put 58% of its cost into context and only 3% into planning. It finished all 5 tasks for $0.45. Sonnet 5.5 and Opus 5.5 spent 40% and 38% on context and about a quarter on planning. Opus cost $1.58 (~3.5x Sol). Sonnet cost $0.72 (~1.6x Sol).
3
3
12
733
Replying to @linear
@linear please tell me labelling and assignments are going to get better with Jev
1
195
me every time a new frontier model drops
2
1
3
428
never tell an agent to "be thorough" we found a "be thorough" in an internal agent system prompt, removing it decreased spend by 67% with no significant change to quality 🤯
1
1
15
813
fat stacks 🥞
1
8
492
what code review looks like in 2027
10
1,135
one of the biggest challenges of agent knowledge MCPs is showing the impact of it to users if done right, memory just works and makes your agent better, but its invisible to users. we've been experimenting with allowing agents to give feedback at the end of their sessions on how @dosu_ai impacted their trajectory: "qualitative agent feedback" the results are early, but really interesting!
1
6
532
Replying to @swyx
I ran out of 500GB of disk space for this same reason
1
1
15
1,947