Chief Scientist @ Jellyfish. Formerly Harvard, MIT Lincoln Lab. I love running; it doesn't always love me back.
Boston, MA
Joined August 2013
- Tweets1.3K
- Following613
- Followers458
- Likes1.2K
🚨 New @_jellyfish_co AI Engineering Trends are out!
TL;DR: software factories, not a thing yet, maybe soon?
Median company: 28% agentic PRs, 0.5% agentically merged
But the top 1% of companies went from 1.6% to 9.4% factory PRs since March.
jellyfish.co/ai-engineering-…
Devs in @_jellyfish_co data letting Claude Code auto approve every edit
April: 4%
September: 34%
PRs merged without a human reviewer? 0.5%
Robots are coding like crazy and devs can't keep up
Want to hear more? Come join me at @tessl_io @ainativedev Con in NYC next week!
Register for aiDevCon here, use code NICKA30 for 30% off: tessl.io/devcon
@devopsdotcom coverage:
devops.com/report-claude-cod…
The AI model conversation is like listening to people argue about wine, half the people debating the subtle notes that make one vintage superior to another, half want to know how drunk it'll get them and how much it's gonna cost
The true beauty of Claude is that years of digital OCD are finally paying off e.g. I can ask and answer upsetting questions like this
It’s a poor golfer who blames their clubs. In my full @aiDotEngineer NYC talk, I use @_jellyfish_co data to show AI engineering works the same way.
It’s not the model, it’s you.
youtube.com/watch?v=Ve2bAarP…
Have a buddy who’s been doing library web apps at a university for 20 years, asked him how he’s doing with all the security chaos this month, this was his response
LLMs: billion-parameter sci-fi algorithm
Skills: glorified folder of canned prompts
LLMs: explain <1% of PR throughput
Skills: explain 3% of PR throughput
(from my @aiDotEngineer NYC talk, cc @_jellyfish_co)
It's kind of insane to think that the typical software org isn't willing to trust 20% of their dev capacity to merge code 🤯
Check out my piece for @tessl_io on the gap we see in the @_jellyfish_co data between agentic coding vs "agentic shipping":
tessl.io/blog/there-arent-en…
Talking about all this at @ainativedev Con in Brooklyn, Nov 2–4. Use code NICKA30 for 30% off: aidevcon.io/
Nicholas Arcolano retweeted
Every time a new frontier model comes out, we go through the same rituals: benchmarks get reported, leaderboards get reshuffled, and the social feeds fill up with a heady cocktail of vibes, hot takes, and everything we’ve found the new model can “one-shot.” Anyone watching from the sidelines could be forgiven for thinking that model performance is the single biggest factor driving AI transformation today.
But day to day, how much does the model really matter?
As part of his work at Jellyfish, Head of Research and AI Nicholas Arcolano analyzes and extracts industry insights from our massive collection of software engineering workflow data, which tracks AI coding tool and agent telemetry across 300,000 developers from 1,300 companies. And when you look at what actually happens inside real engineering teams after a new model drops, the picture is a lot more complicated than the launch-day discourse would suggest.
He shared insights during AI Engineer NYC last week. See an overview of "It's Not the Model, It's You" here: dev.to/dailycontext/its-not-…
Nicholas Arcolano retweeted
The plants all around us are filled with bio-electrical signals that can tell us about the plant, its health, and the environment that we share. What if we could listen?
That's why we built Bombadil - our foundation model for decoding plants' electrical language.
Stop arguing about which frontier model is better, it doesn't matter
(Clip from my @aiDotEngineer NYC talk – cc @_jellyfish_co)
Still annoyed with Claude for embarrassing me in front of @dexhorthy at @aiDotEngineer last week.
Same feeling as when you try to show off your well-behaved kids at dinner and they choose to act like insolent little brats instead.
Nicholas Arcolano retweeted
To make rust build times faster, key to rust's viability as an agentic coding language, I have asked Claude to rebuild it in python
Just made it to @aiDotEngineer, first thing I saw was my story in "The Daily Context". Haven't had a story in a print newspaper since high school 😆
Check it out! @_jellyfish_co insights about what what happens when devs change LLM model families.
dev.to/dailycontext/its-not-…
