taesiri retweeted
finally releasing our new OCR benchmark
tons of hard examples like this mechanical dial meter
- single value extraction
- JSON data extraction
- document transcription
- text localization and recognition
- 48 models evaluated
↓ examples and leaderboards
taesiri retweeted
We gave GPT-6.1 Sol a pen✍🏻
Try our public SketchVLM demo: ask a question, and it draws on your screen or image to explain its answer and guide you!
🌟 Accepted to EMNLP 2026 Main!
Demo: sketch-vlm-demo.vercel.app/
Project: sketchvlm.github.io/
taesiri retweeted
Opus 5.5 (max) is no.2 on ZeroBench
43% pass@5
25% pass^5
Now, 77% of questions have been solved at least once
It is impressive how much progress has been made in so little time... from VLMs being blind to outperforming humans on ZeroBench.
GPT-6 Astra (max) on ZeroBench:
pass@5: 51% (prev. SOTA: 30%)
pass^5: 35% (prev. SOTA: 13%)
If this reflects a genuine capability improvement, it is a significant step forward in both performance and consistency
1
20
522
GitHub just published they spent~$120K for their TypeScript to Rust port. My estimate for something at this scale was ~$25K–$160K.
github.blog/ai-and-ml/genera…
Tried to measure how many tokens are actually inside some well-known repos, and put some very rough estimates on what it'd cost to make them.
Linux kernel is ~430M tokens, and would cost between an unrealistic floor of ~$13K and an upper bound of ~$5.4M if you built it with GPT-5.6 Sol.
The cost numbers are pretty silly honestly, it's only good for having some mental picture of the scale.
taesiri.xyz/TheTokenCensus/
2
9
573
Today, this sample from @OpenAI GPT Image 2.5 caught my attention. Almost exactly a year ago, we ran similar stress tests, and image models were nowhere near this level in terms of subject consistency and image quality.
One year later, this feels almost fully solved.
With all the attention on the new 🍌 Nano Banana, we (with @brandon_co92810) stress-tested it with our "shirt-color chain" benchmark: 26 sequential edits where each output feeds into the next.
Learn more our study on image editing models here: psrdataset.github.io/
#nanobanana #AI #images
Everyone is coming up with their own benchmark, so I came up with mine: make a demonstration video showing how to prepare Salad Shirazi.
Verdict: GPT-6 Astra is not AGI yet 😂
While, I appreciate the ingredients, especially the verjuice, but those slices are sooo thick. Not approved by any of my friends from Shiraz :)))
shirazi-kitchen-motion.taesi…
1
8
642
Here is my first GPT-6 Astra test using CUA and Blender. Asked it to build a 3D model of the Azadi Tower. It passes a rough visual check from a distance, but it's not quite accurate and misses some shape details and proportions. Still, it's impressive that this is possible with a single prompt.
1
2
495
taesiri retweeted
Accepted to EMNLP Main!
Hood popped. Photo taken. “Hey ChatGPT, how do I check my car’s oil level?” And it returns a giant block of text…
Instead, a human would point to the oil cap and draw on the photo to answer! We explore how to unlock VLMs to do that, i.e., annotating on the image to guide users through answers visually: sketchvlm.github.io/
1/n 🧵
Tried to measure how many tokens are actually inside some well-known repos, and put some very rough estimates on what it'd cost to make them.
Linux kernel is ~430M tokens, and would cost between an unrealistic floor of ~$13K and an upper bound of ~$5.4M if you built it with GPT-5.6 Sol.
The cost numbers are pretty silly honestly, it's only good for having some mental picture of the scale.
taesiri.xyz/TheTokenCensus/
Gave my Claude a face… and I immediately regretted it 🙈🙈 Easily the scariest thing I've built so far.
My team at Google XR just started open-sourcing GNM, our ecosystem of parametric human models and perception stacks, starting with GNM Head.
Exciting to see the community adopt and experiment with it so enthusiastically! 🥳
xitter.gq/soyposmoderno/status/2…
Sol Ultra is a monster at reproducing research papers.
Gave it an empty folder and just two PDFs of research papers on fire simulation, with no code or markdown, and asked it to recreate the project with WebGPU.
A few hours later … here is the results:
fire-replica-lab.taesiri.cha…
13
417
Watched Backrooms and wanted to test Luna again. Building an endless procedural world in Unreal with code, not MCP, plus a VHS camera and an attention-driven wanderer that moves and looks like a scared human. At least, that was the goal lol. As usual Sol did a bit of help.
Luna with high reasoning is seriously underrated.
I revived one of my PhD projects with just a few prompts to use modern models; a live 3D dashboard for testing how VLMs from CLIP to Gemma 4 respond to different viewpoints.
It imports 3D meshes directly from Sketchfab, runs inference, generates 3D heatmaps, and checks the plausibility of the answers using a local text-only Qwen model.
Sol helped a bit with the video export.
Here is the link to the Sketchfab model sketchfab.com/3d-models/tomm…
180
My excitement for Fable 5 keeps growing 🤩
I tested whether it could port a terrain erosion simulation from C++/CUDA to WebGPU.
It was't one-shot magic, but it got there… sort of :D
github.com/taesiri/soilwebgp…


















