Dev, AI engineer @02AIDEV https://xitter.cf/t.co/BlqJIbUN2n 3rd placer 🏆 1st ever Web3 vibe coding hackathon 2025 | Vibed 9 games in 3 weeks 🎮 |
Build your AI with us →
Joined September 2023
- Tweets23.3K
- Following1.6K
- Followers3.6K
- Likes72K
Pinned Tweet
Pi Agent vs OpenCode token usage
A lot of people recommended Pi Agent so I decided to check
Pi Agent took 1.1k tokens in first turn
OpenCode took 11.5k
Setup:
1) Trimmed OpenCode (from usual 30k first turn to 11.5k)
- 0 MCPs
- 2 lightweight plugins (opencode-env-protect and openslimedit)
- 8k char AGENTS.md
- 11585 deepseek-v4-flash input tokens
- for $0.0016
2) Vanilla Pi
- 0 MCPs
- 0 system prompt
- 8k char AGENTS.md
- 1114 kimi-k2.6 input tokens
- for $0.0008
I think using better models with capped tokens per turn can keep usage nearly the same as uncontrolled DeepSeek V4 Flash?
The challenge now is finding the sweet spot. Can't cap tokens if quality drops.
We'll see.
Video below - OpenCode vs Pi side by side "say hi back" first turns test, with OpenCode Go usage for each
ego (lite) is probably the most underrated browser agent atm
my usual multi-step QA task that will take browser use multiple trial and errors in python and takes cua driver 3x as long, is executed flawlessly in ego lite
however i also think it's unfair to compare ego lite to cua and browser use since ego has a closed source browser which does the heavy lifting bts
simply comparing apples to oranges
and whatever ego is doing bts with their browser, where all the magic lies, is a blackbox
atm what's on their github is just their sdk and skills
these hacking agents are getting more sophisticated but look, even your deepseek agent can easily bypass a regex guardrail
simply ask it to creatively bypass patterns when it hits a wall, and it will gladly do so
guess what it'll try first?
a classic programming trick taught to beginners: string concatenation
seen it pursue restrictions so many times as if it were a game
lol im taking it back
deepseek is apparently this good
xitter.cf/ValsAI/status/21124710…
We found DeepSeek V4.1 Flash 1017 beats GPT 6 Sol on our multi-step offensive cybersecurity task. Sol was tested via the Daybreak trusted access program.
We have been building a cybersecurity benchmark based on simulated corporate networks. This is a valuable indicator for the cyber-risk a model poses if in the hands of a bad actor. We plan to make the full evaluation public in the coming weeks but wanted to share our first immediate finding here.
I like this take. Time for Claude to have a proper mobile app and Duet might be it
a deepseek refresh also means peak api speed
2 research agents off peak hours:
35.4k tokens in 47s
44.8k tokens in 45s
my oc set up kickstarts subagents at 7-8k tokens
at ~80% output tokens:
602.6 TPS
796.4 TPS
could have been faster if the task did not involve webfetch
DeepSeek V4.1 Flash 1017 scores 45 on the Artificial Analysis Intelligence Index, improving on DeepSeek V4.1 Flash by 6 points, and is the equal #2 open-weights model
@deepseek_ai has released DeepSeek V4.1 Flash 1017, an update on their flagship model DeepSeek V4.1 Flash. On the first-party API, DeepSeek V4.1 Flash 1017 is priced at $0.30 per 1M input tokens and $1.20 per 1M output tokens during peak hours, with off-peak pricing half of peak rates, the same pricing as DeepSeek V4.1 Flash.
Key results:
➤ DeepSeek V4.1 Flash 1017 makes gains in agentic capabilities, knowledge and scientific reasoning. DeepSeek V4.1 Flash 1017’s Elo rating on GDPval-AA v2.1 is 1746, an increase of 146 points from DeepSeek V4.1 Flash’s rating of 1600. On AA-Briefcase v1.1, its Elo rating increases by 150 points from 1420 to 1570. On AA-Omniscience, its non-hallucination rate is 33%, a 29-point improvement from DeepSeek V4.1 Flash’s 4%, while seeing an improvement in accuracy from 43% to 46%, pushing its overall AA-Omniscience Index score from -5 to +5. On CritPt, DeepSeek V4.1 Flash 1017 scores 24%, up from 14% for DeepSeek V4.1 Flash.
➤ DeepSeek V4.1 Flash 1017 is now the #2 open-weights model alongside GLM-5.3. With a score of 45 on the Artificial Analysis Intelligence Index, DeepSeek V4.1 Flash 1017 is behind MiMo-V2.6-Pro by only 1 point and ahead of other open-source models like Step 5 Preview and Kimi K3 (both at 44).
➤ With a Cost per Task of $0.26, DeepSeek V4.1 Flash 1017 sits in the most attractive quadrant of Capability vs. Cost. With a Cost per Task of $0.26 and Intelligence Index score of 45, DeepSeek V4.1 Flash 1017 compares favorably to GLM-5.3-Flash at $0.28 per task and a score of 42. At the same time, MiMo-V2.6-Pro (46) at $0.13 per task and GPT-6.1 Sol (medium, 48) at $0.21 per task both offer higher capability at a lower Cost per Task.
Additional model details:
➤ Context window: 1M tokens (equivalent to DeepSeek V4.1 Flash)
➤ Pricing: $0.30 per 1M input tokens and $1.20 per 1M output tokens, unchanged from DeepSeek V4.1 Flash. Cache hit price of $0.006 per 1M tokens, a 98% discount
➤ Size: 552B total parameters (16B active)
➤ License: MIT
➤ Providers: Available through DeepSeek’s first-party API, @FireworksAI_HQ , @baseten, @DeepInfra , @wafer_ai , @Cloudflare and others
so it's their secret sauce
xitter.cf/deepseek_ai/status/211…
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
obviously a bad test since it's a research task involving webfetches
and output tokens were estimated
but the eye test shows how godly fast the api is
Looks like spending a weekend to strip your tools of telemetry and anything that you don't want sent to remote servers will become more common
Thanks to REA and fleets of AI agents getting good at finding things
Recently did that with a tool I'm trying out
A reminder why enterprise cloud needs training that even configuring complex IAMs and provisioning service accounts require strict guardrails
Has been the case before hackers were empowered by fleets of AI agents. Which means guardrails has to evolve in record velocity as more of these attacks happen here and there
IT trained people would tell you to have a sense of trusting only to the incumbents with plenty of resources and enterprise track record with plenty to lose
Add the overwhelming "bureaucratic" processes inertwining people and systems, the layers of security, and the fleet of L1-N engineers these incumbents have
Not saying these incumbents with massive resources can't be compromised, but it is surely harder and more intimidating to do so, at least on paper
holy shit, opencode got hacked and it's officially fu*ked
'a large scale distillation attack, 1.4% of console users hit'
and people got drained, one user says his openai and anthropic keys, used only in opencode, ran up over $400 overnight, another replied $1,700. who compensates that?
every one of them had external providers linked, and it's still ongoing
this is the warning for all of us, build your own harness and keep your keys on your own machine, don't park them in anyone's console anymore
I'm very happy with Paseo, which has been my daily driver for months already
But since T3 Code is in the buzz lately due to their in-app visualization capabilities, I decided to play around with it
First thing I noticed is how much work was put into rendering output elements
If you spend time reading agent outputs you'll notice how improved reading experience matters in long work sessions, let alone visual experience
The overall interface is also intuitive. Despite being packed with features, it's surprisingly easy to find your way around.
One tiny quirk is not seeing reasoning blocks. Probably exclusive to OpenCode integration, as I didn't try connecting other harnesses yet
I also have to see how it'll handle hundreds of sessions in which Paseo, my daily driver, handles with ease on a desktop/laptop but very buggy on mobile
I'll definitely track T3 Code as the potential is there
We just shipped in-app visualization capabilities in T3 Code. This enables agents to build cool dynamic experiences within the thread.
Huge shoutout to @davis7 for pushing us on this (and building most of it). Didn't expect to dig it so much.
Personal experience with GLM 5.3 Flash is it will miss tiny details once it hits 200k context
Interesting how it deals with legal workflows which requires massive context
It's a sound model and reasons a lot but often lacks the foresight of a heavily aligned frontier model
Introducing GLM-Legal
Built on GLM-5.3-Flash, GLM-Legal matches frontier models on mainstream legal benchmarks.
Designed for real legal workflows: contract review, evidence analysis, compliance, and more.
Tech Blog: z.ai/blog/glm-legal
Apply for API Access: z.ai/consultation/legal
Try AutoClaw: autoclaw.z.ai
I mostly use uv and nvm but it's becoming really messy now that I increased my rate of spinning experimental projects to thrice a week
thanks to today's sophisticated AI coding agents
add that these agents like running quick inline tests which usually requires a node/python package
that meant friction if there's no planned environment in place and you care about your main builds left unscathed
and once it turns into something serious, it adds into another environment and dependency to manage
looks like mise is a frictionless way to do that so I have to try
especially now that most of my quick experimental projects are now in js + python, and lives inside my local cua computer as well
sounds like mise is a great complementary tool in that stack
Another UX masterpiece. This is how intuitive I expect Claude Code mods to be
Build richer, more interactive plugins for ChatGPT.
With plugin extensions, you can launch plugins from the sidebar, support @ mentions, and add file viewers.
See how in this walkthrough with @coreyching:
It looks like that they also implemented the same experience in ChatGPT with the same core in plugin extensions
But what's notable here is how intuitive it is
xitter.cf/OpenAI/status/21078949…
Sonnet is supposed to be Anthropic's Flash model but look at Claude Haiku 5.5
Anthropic has released Claude Haiku 5.5, scoring 43 on the Artificial Analysis Intelligence Index - up 26 points one year after the last Haiku release
Haiku 5.5 is the first Haiku model with Anthropic’s effort settings and adaptive thinking, and Anthropic has introduced tiered pricing.
Haiku 5.5 is cheaper than its predecessor - it costs $0.10/$0.50 per 1M input/output tokens for prompts up to 100k tokens (the same as GPT-6 Luna and 10% of the previous Haiku model). However, this pricing rises 5x to $0.50/$2.50 above 100k. The site does not yet reflect tiered pricing, so provisional cost figures for Haiku 5.5 do not include the step up cost. We are working on support and will follow up with Cost per Task coverage soon.
Key takeaways:
➤ Leading small-class model performance: At max effort Haiku 5.5 sits slightly ahead of models such as GLM-5.3 Flash (42), Gemini 3.8 Flash (41) and GPT-6 Luna (38). Its score is comparable to Kimi K3 (44), a 2.8T parameter open weights model, and trails Claude Sonnet 5.5 (max, 56) by 13 points
➤ Heavy token use compared to GPT-6 Luna: Haiku 5.5 (max) uses ~162k output tokens per Intelligence Index task, ~3x GPT-6 Luna (max, ~50k). Moving from xhigh to max adds 2 points for ~1.8x the tokens. At similar intelligence it also uses more tokens than GPT-6 Luna: Haiku 5.5 (high) scores 38 with ~55k tokens per task against 38 with ~50k for Luna (max), and the gap widens at lower effort settings
➤ Highly capable at agentic knowledge work: on AA-Briefcase, our private evaluation for realistic knowledge work tasks, Haiku 5.5 (max) reaches 1578 Elo, ahead of models including Kimi K3 and GLM-5.3, and comparable to Muse Spark 1.3 (max)
➤ Improvements on terminal use: on Terminal-Bench 4.0 it scores 33%, up from 0% for Haiku 4.5. This is level with GLM-5.3 Flash, and ahead of Gemini 3.8 Flash (20%) and GPT-6 Luna (13%)
➤ Lower factual knowledge, but relatively low hallucinations: as expected for a smaller-class model, Haiku 5.5 has lower factual knowledge than its siblings. AA-Omniscience accuracy is 36%, against 55% for Gemini 3.8 Flash and 44% for GPT-6 Luna, but this is partly driven by more willingness to admit when it doesn’t know - its hallucination rate is lower, at 40% against 55% and 77%
➤ AutomationBench-AA result likely understated: Haiku 5.5 scores 35%, against 53–60% for GPT-6 Luna, Gemini 3.8 Flash and GLM-5.3 Flash. During pre-release testing, a safety refusal issue caused the model to over-refuse. Anthropic is working on resolving this - we will re-run this evaluation with the fix, and expect this score to rise
Other model details:
➤ Context window: 1 million tokens, up from 200k for Claude 4.5 Haiku
➤ Pricing: $0.10/$0.50 per 1M input/output tokens up to 100k tokens, $0.50/$2.50 above. Cache reads $0.01 ($0.05 above 100k), 5 minute cache writes $0.125 ($0.625 above 100k)
➤ Multimodality: Text and image input, with text output
