Concept (@conceptdotdev) header
Concept (@conceptdotdev) profile picture
Joined January 2026
We launched companyagentbench.com to measure how well agents understand specific companies. Read the results here:
Today we鈥檙e releasing Company Agent Bench, a benchmark for company assistant products. We compare Dots, Grok Bot, Claude Tag, and Devin on how well an agent understands its company. We also found that API models with a simple harness perform better than these products (1/6)馃У
8
367
Concept retweeted
Today, we鈥檙e launching companyagentbench.com @OpenAI Dots placed #1 across agent products while @AnthropicAI Fable 5.2 placed #1 overall via API. Our goal is to measure how well agents understand a specific company. Check out the results and findings below! Shoutout @joshzqin for spearheading this project
Today we鈥檙e releasing Company Agent Bench, a benchmark for company assistant products. We compare Dots, Grok Bot, Claude Tag, and Devin on how well an agent understands its company. We also found that API models with a simple harness perform better than these products (1/6)馃У
23
6
1
151
12,619
Concept retweeted
Today we鈥檙e releasing Company Agent Bench, a benchmark for company assistant products. We compare Dots, Grok Bot, Claude Tag, and Devin on how well an agent understands its company. We also found that API models with a simple harness perform better than these products (1/6)馃У
24
23
7
212
28,415
Concept retweeted
I think there's an interesting tension in the market rn. Agents only really started to work about a year ago with Opus 4.5. Many vertical AI companies since have gone exponential in growth (just chart out the valuation and revenue numbers YTD). At the same time, when talking to enterprises, they aren't feeling a truly MAGICAL wow that agents have delivered significant business impact. So what's the actual gap between agents that can solve millennium prizes and delivering real business impact? My cofounder Phil who was a researcher at OAI/GDM has been thinking deeply about this problem and talks through where agents fail today and the next frontier research problems to close this gap.
6
2
2
64
6,651
Concept retweeted
The Frontier or Nowhere We鈥檙e hiring engineers. Come work with me & @philhchen
21
2
2
83
11,237
Concept retweeted
Gemini 4 Argon reaches Astra-level performance on multimodal data annotation. Significant improvement compared to Gemini 3.8. Argon also better at identifying correct actions in egocentric videos.
Today, we're sharing results from our partnership with @MeckaAI to see if frontier models can label egocentric videos for robot learning. We had five frontier models caption 200 clips with human grading. GPT-6 Astra which performed the best got 43.5% of clips fully correct.
4
1
1
28
1,974
Incredible model, great to partner
1
143
Great to collab on this!
4
319
Replying to @MeckaAI @tkkong
Let's go! Great to partner with the team
18
Concept retweeted
Frontier models still struggle to accurately caption egocentric videos. Dense video captioning remains a task where a specialized model, our collaboration with @MeckaAI, outperforms general-purpose foundation models. Many interesting problems still to solve in robotics.
Today, we're sharing results from our partnership with @MeckaAI to see if frontier models can label egocentric videos for robot learning. We had five frontier models caption 200 clips with human grading. GPT-6 Astra which performed the best got 43.5% of clips fully correct.
6
3
1
49
3,337
Sharing our first research post with @MeckaAI
Today, we're sharing results from our partnership with @MeckaAI to see if frontier models can label egocentric videos for robot learning. We had five frontier models caption 200 clips with human grading. GPT-6 Astra which performed the best got 43.5% of clips fully correct.
3
1
21
1,480
Concept retweeted
We鈥檙e hiring for designers & engineers. Come work with me & @philhchen
86
7
1
439
32,512
Concept retweeted
We're opening up early access. Phil and I have been working with teams to ship internal AI apps and agents. We鈥檙e applying our learnings from Ramp Labs, OpenAI, and DeepMind to help operations teams scale faster. DM if you're interested in trying it out!
30
8
5
175
41,838