Organizing human intelligence to power the AI economy.
San Francisco
Joined April 2021
- Tweets473
- Following31
- Followers26.3K
- Likes865
Pinned Tweet
Workers in six occupations do 79 tasks the government's official occupational taxonomy has yet to add. That work is worth about $140B a year in wages.
That taxonomy is called O*NET, the U.S. Department of Labor's database of the tasks each occupation involves. Workers use it to help with job transitions, and economists use it to estimate which jobs AI will change. If the lists are incomplete, so are the estimates.
Today we're releasing TaskNet, an open-source task catalog built on O*NET and updated for how AI is changing work.
Built with researchers from Stanford, the O*NET Resource Center, and NBER. Thank you to @robinsonaisaac @erikbryn @diyi_yang @jennjwang @andreas_h0wpt @Digeconlab @stanfordnlp @stanfordhai @stanfordAIlab for contributing to this work.
Read the blog: mercor.com/blog/introducing-…
Mercor retweeted
Mercor's @edwardjhu reveals why researchers are designing AI benchmarks with tasks even humans can't solve, just to make models fail:
"The benchmarks we have today are not gonna be good benchmarks in a year or two or even less."
"As the models become more capable, it’s increasingly hard to fail the model. And there is this tendency to fail the model at all costs, often at the expense of fairness or benchmark diversity."
"We’re asking the AI to do something that’s actually impossible, a human might not even be able to do it, or we find one way the model fails and we replicate it so many times"
"If the entire benchmark is testing just one failure mode, the benchmark score will swing from zero to near 100% very quickly. It’s easy for someone to look at a number and draw conclusions right away."
"But we really need to be thoughtful about the underlying design so that it’s not just a number people can cite, but a reliable instrument for the field."
@mercor
GPT-6.1 Terra is OpenAI’s #1 model on APEX-Agents.
68.3% Pass@1 (#6)
79.3% mean score (#6)
Compared to last generation GPT-5.6 Terra, the new model gains 10.1 points on Pass@1.
Management consulting: 80.9% Pass@1 (#2)
Investment banking: 66.2% Pass@1 (#7)
Corporate law: 57.7% Pass@1 (#39)
The model is strongest on management consulting tasks, ranking second overall, behind Gemini 4 Argon (88.4%).
Against GPT-5.6 Terra, it gains 24.7 points in consulting and 10.9 in banking. Law goes the other way, down 5.4 points from GPT-5.6 Terra.
GPT-6.1 Terra still averages 80.6% of rubric criteria on law tasks, but fully completes only 57.7% of them. It gets most of the way and misses the last requirements.
Across 4 runs per task, GPT-6.1 Terra passed 134 of 240 tasks every time and 187 at least once. 22 tasks it never scored on, 14 of them in investment banking.
GPT-6.1 Terra’s strength comes from reading more than its siblings: 735k input tokens per attempt, vs 497k for GPT-6 Astra and 489k for GPT-6.1 Sol. Despite reading more, it costs less.
GPT-6.1 Terra: $0.19 per task ($1/$5 per 1M tokens)
GPT-6.1 Sol: $0.26
GPT-6 Astra: $1.30
It’s also a fast model. The median attempt finishes 1.2x faster than GPT-6 Astra and on par with GPT-6.1 Sol.
Congrats @OpenAI on the launch.
See full leaderboard: mercor.com/apex/apex-agents-…
Mercor retweeted
I will be live on MTS at 1:15pm PT to talk about Mercor Research, AI for economically valuable work, model training for enterprise, and more.
OPENAI PHYSICS BREAKTHROUGHS | BLUECHIP LAPTOP | QWEN NEURONS xitter.cf/i/broadcasts/1nGeLMNqg…
Join us tomorrow in San Francisco for a free scoop of @Mercor mocha ice cream.🍦
Congratulations @JudgmentLabs on releasing general access and thanks for the sweet treat.
500 Howard St.
San Francisco
Wednesday, October 28, 11AM-1:15PM
Mercor retweeted
We used Mercor's AI interviewers to build a new catalogue of economically valuable work in the U.S with @Stanford.
We found that O*NET, the current standard, misses about a third of the tasks and ~$140B in annual wages.
Workers in six occupations do 79 tasks the government's official occupational taxonomy has yet to add. That work is worth about $140B a year in wages.
That taxonomy is called O*NET, the U.S. Department of Labor's database of the tasks each occupation involves. Workers use it to help with job transitions, and economists use it to estimate which jobs AI will change. If the lists are incomplete, so are the estimates.
Today we're releasing TaskNet, an open-source task catalog built on O*NET and updated for how AI is changing work.
Built with researchers from Stanford, the O*NET Resource Center, and NBER. Thank you to @robinsonaisaac @erikbryn @diyi_yang @jennjwang @andreas_h0wpt @Digeconlab @stanfordnlp @stanfordhai @stanfordAIlab for contributing to this work.
Read the blog: mercor.com/blog/introducing-…
Mercor retweeted
Understanding the future of work is a pressing problem, but in order to do so, we need to understand and measure the present. I'm proud to have helped lead this collaboration between academia, government, and industry to produce TaskNet - an occupational taxonomy that will help us better understand how work is changing.
The future of work is a shared responsibility that should not be taken lightly.
Workers in six occupations do 79 tasks the government's official occupational taxonomy has yet to add. That work is worth about $140B a year in wages.
That taxonomy is called O*NET, the U.S. Department of Labor's database of the tasks each occupation involves. Workers use it to help with job transitions, and economists use it to estimate which jobs AI will change. If the lists are incomplete, so are the estimates.
Today we're releasing TaskNet, an open-source task catalog built on O*NET and updated for how AI is changing work.
Built with researchers from Stanford, the O*NET Resource Center, and NBER. Thank you to @robinsonaisaac @erikbryn @diyi_yang @jennjwang @andreas_h0wpt @Digeconlab @stanfordnlp @stanfordhai @stanfordAIlab for contributing to this work.
Read the blog: mercor.com/blog/introducing-…
Mercor retweeted
A labor of love with Mercor and the O*NET Center! TaskNet brings together academic research, industry experience, and occupational data expertise to turn workers’ experiences into task information that can inform workforce policy and training.
Workers in six occupations do 79 tasks the government's official occupational taxonomy has yet to add. That work is worth about $140B a year in wages.
That taxonomy is called O*NET, the U.S. Department of Labor's database of the tasks each occupation involves. Workers use it to help with job transitions, and economists use it to estimate which jobs AI will change. If the lists are incomplete, so are the estimates.
Today we're releasing TaskNet, an open-source task catalog built on O*NET and updated for how AI is changing work.
Built with researchers from Stanford, the O*NET Resource Center, and NBER. Thank you to @robinsonaisaac @erikbryn @diyi_yang @jennjwang @andreas_h0wpt @Digeconlab @stanfordnlp @stanfordhai @stanfordAIlab for contributing to this work.
Read the blog: mercor.com/blog/introducing-…
We asked workers whether they could define success on each task in advance and if they could easily check for success after the task was done. For knowledge workers, the answer was often no to both.
ER physicians said no to at least one of these questionsfor 60% of their tasks, like managing patient flow through the ER. Developers said no for 50% of their tasks and lawyers for 46%.
For dishwashers, the share was 11%.
This matters particularly for AI. To train AI with reinforcement learning, you need a way to score its outputs.
A special thank you to Phil Lewis for advising on this project and to all the Mercor experts who made this research possible.
Full paper: papers.ssrn.com/sol3/papers.…
Blog post: mercor.com/blog/introducing-…
Project page: digitaleconomy.stanford.edu/…
GPT-6.1 Terra is OpenAI’s #1 model on APEX-Agents.
68.3% Pass@1 (#6)
79.3% mean score (#6)
Compared to last generation GPT-5.6 Terra, the new model gains 10.1 points on Pass@1.
Management consulting: 80.9% Pass@1 (#2)
Investment banking: 66.2% Pass@1 (#7)
Corporate law: 57.7% Pass@1 (#39)
The model is strongest on management consulting tasks, ranking second overall, behind Gemini 4 Argon (88.4%).
Against GPT-5.6 Terra, it gains 24.7 points in consulting and 10.9 in banking. Law goes the other way, down 5.4 points from GPT-5.6 Terra.
GPT-6.1 Terra still averages 80.6% of rubric criteria on law tasks, but fully completes only 57.7% of them. It gets most of the way and misses the last requirements.
Across 4 runs per task, GPT-6.1 Terra passed 134 of 240 tasks every time and 187 at least once. 22 tasks it never scored on, 14 of them in investment banking.
GPT-6.1 Terra’s strength comes from reading more than its siblings: 735k input tokens per attempt, vs 497k for GPT-6 Astra and 489k for GPT-6.1 Sol. Despite reading more, it costs less.
GPT-6.1 Terra: $0.19 per task ($1/$5 per 1M tokens)
GPT-6.1 Sol: $0.26
GPT-6 Astra: $1.30
It’s also a fast model. The median attempt finishes 1.2x faster than GPT-6 Astra and on par with GPT-6.1 Sol.
Congrats @OpenAI on the launch.
See full leaderboard: mercor.com/apex/apex-agents-…
Mercor retweeted
We’ll pay you up to $100K for every company you refer that licenses its data to @Mercor.
We’ve partnered with > 500 companies so far.
If you know a company with 100+ employees and data in code, gaming, manufacturing, healthcare or professional services, refer them at data.mercor.com/refer.
Mercor partnered with @Reflection_ai on Beam's training and evaluation.
Beam scores 48.8% Pass@1 on APEX-Agents, #1 among US open-weight models and closely trailing Chinese open models such as GLM-5.3-Flash (52.8%).
On APEX-Agents, which measures long-horizon, multistep tasks in professional services, Beam scores a 15-point lead among US open-weight models.
Beam: 48.8%
Muse Spark Open: 33.4%
Nemotron 3.5 Super VL: 29.5%
Nemotron 3 Ultra: 22.7%
It trails Qwen 3.8-Max by 14.5 points and lands within the confidence interval of GLM-5.3-Flash, trailing by 4.0 points.
Qwen 3.8-Max: 63.3%
MiMo-V2.6-Flash RL: 57.4%
GLM-5.3-Flash: 52.8%
Beam: 48.8%
Kimi K3: 47.8%
Qwen 3.8-27B: 47.5%
GLM-5.2: 45.2%
Compared with GLM-5.3-Flash on APEX-Agents domains, it leads on management consulting by 1.5 points. It trails GLM-5.3-Flash by 11.8 points on corporate law, and investment banking by 1.6 points.
Congratulations to the Reflection team. We’re excited for future collaboration together advancing the US open source frontier.
See full leaderboard: mercor.com/apex/apex-agents-…
We’ve paid out $50M+ to businesses for their data. Now we’re paying for introductions too.
Refer a company to Mercor and earn up to $100K when they partner with us.
Refer here: bit.ly/4sk28EA
Mercury 3, @_inception_ai's diffusion LLM, debuts on the APEX leaderboards.
50.2% Pass@1 on APEX-Agents (#30 of 59)
43.1% Pass@1 on APEX-SWE (#31 of 62)
Inception positions Mercury 3 against speed-optimized frontier models.
On APEX-Agents it lands between GPT-6 Luna and Haiku 5.5, and on coding it trails Haiku 5.5 by 12 points.
Across APEX-Agents domains (mean score):
Corporate law: 78.4%
Management consulting: 57.5%
Investment banking: 51.4%
It is cost pareto optimal on APEX-Agents. An average APEX-Agents task costs about 6 cents at Inception's list price, and about 3 cents at the launch discount, helped by heavy prompt caching.
Congratulations to the Inception team on showing a diffusion model can hold its own on long-horizon professional work.
See full leaderboard: mercor.com/apex/apex-agents-…
Last week, Mercor Research economist @aden_barton joined @MTSlive to discuss our latest study comparing frontier AI models with licensed accountants on real-world accounting tasks.
18 months ago, the best models performed below the average accountant. Today, models are faster and more accurate than every accountant in our study.
Aden discussed what this progress means for the accounting profession, and why performing well on individual tasks is different from doing an entire job.
Watch the full conversation. youtube.com/watch?v=9JcfR1Lb…
Mercor retweeted
"The entire way that humans contribute to the economy is going to move.”
@mercor Co-founder & CEO @BrendanFoody explains what he believes will replace repetitive human work.