AI safety research @thinkymachines. Formerly @mldcmu @penn @swarthmore
San Francisco, CA
Joined July 2020
- Tweets773
- Following1.6K
- Followers2.1K
- Likes27K
Pinned Tweet
Chatbots like ChatGPT can be jailbroken to output harmful text. But what about robots? Can AI-controlled robots be jailbroken to perform harmful actions in the real world?
Our new paper finds that jailbreaking AI-controlled robots isn't just possible.
It's alarmingly easy. 🧵
Alex Robey retweeted
It’s a good algorithm sir
Off-policy RL is difficult; runs can suddenly collapse because of async RL staleness or numerical mismatch.
StableReinforce is a new method on Tinker that keeps your RL runs stable, even when learning drifts from sampling.
tinker-docs.thinkingmachines…
Alex Robey retweeted
Dimuon is a new optimizer we built for LoRA fine-tuning. In our SFT tests, it often reaches lower loss in fewer steps than AdamW. We're still experimenting with Dimuon; you can Tinker with it today and share your feedback!
See our optimizers guide at tinker-docs.thinkingmachines…
Alex Robey retweeted
We're recruiting PhD (DPhil) students at @UniofOxford to tackle some of the most important open problems in technical AI safety and governance!
In our Technical Safety & Governance (TSG) Lab, we're particularly interested in developing methods to better understand, evaluate and control advanced AI systems.
Some of the research directions we're excited about include:
Training models to explain themselves and other models: Can we train AI systems to provide explanations that help humans and other AI systems better understand, predict and oversee their behaviour?
Understanding and controlling how model behaviours emerge during training: Can we predict undesirable behaviours from early training checkpoints and intervene before they emerge?
These are just examples of the broader research questions we're excited about. We're open to other ambitious and original ideas in related areas, so don't feel limited to the specific approaches outlined.
There are also exciting research opportunities across AIGI, including projects led by fellow lab directors pro @maosbot on Bayesian governance and Prof. Amro Awad on hardware governance.
Who should apply?
If you have a strong technical background in machine learning, computer science, mathematics, or related fields, and are excited about pursuing ambitious research that could help make advanced AI systems safer, we'd love to hear from you.
I am committed to partnering with researchers from disadvantage and under represented backgrounds.
How to apply:
Explore the example projects, identify a research direction that interests you, and contact the relevant lab director to discuss your ideas. Your application should clearly address at least one of the AIGI labs.
📅 Deadline: 2 December 2026 (Direct DPhil route)
Please share with talented students and researchers who might be interested!
#PhD #AISafety #AIGovernance #Oxford #AIGI
Alex Robey retweeted
Tinkerers have been busy scaling up long-context RL! We’ve made significant improvements to Tinker’s efficiency to support those, and are passing these on with price cuts up to 70%.
GLM-5.3-Flash and DeepSeek-v4.1-Flash are also live for cost-efficient long-context work.
Alex Robey retweeted
Check out our new paper on Inference Auctions! We also design inference auction autobidders, which allow a user to specify an inference budget and dynamically adjusts its bids over time to maximize user utility subject to the budget constraint.
When LLM demand exceeds capacity, which requests should go first?
Model providers already offer a few priority tiers. But even within a tier, requests compete for compute, and users value latency differently. Your coding agent might be blocked due to a request that belongs to an unattended background job.
Why not let the market decide?
In our new paper, Inference Auctions, users bid for request priority, and our auction allocates that priority to those who value it most.
Unfortunately, naively selling to the highest bidders destroys prefix-cache locality, increasing SGLang's average latency by up to 12x in our experiments.
Our fix is to run an auction over depth-first search traversals of the request prefix tree, ordering subtrees by average bid. This maximizes social welfare while retaining cache optimality, with VCG payments incentivizing truthful bidding, and the resulting request order preserves key performance metrics (cache hits and mean TTFT) of the underlying scheduler.
w/ @keegan_w_harris @sidpra5ad @nhaghtal and Michael Jordan
Paper: arxiv.org/abs/2609.40070
Project page: inferenceauctions.com
Alex Robey retweeted
we’re hiring a founding model designer to shape how Inkling behaves and feels.
bring a writer’s precision, a designer’s judgment, and the technical rigor to translate your ideas into data and evals.
please DM your best work or apply below.
jobs.ashbyhq.com/ThinkingMac…
Alex Robey retweeted
In our Nature paper, we introduce the first superhuman Stratego AI, which we built using general techniques that we developed for RL & test-time compute under imperfect information. 1/N
Alex Robey retweeted
AI safety and alignment is more of a data + environment design problem than most realize!
Yes - there is great work to be done on inference time monitoring and controls, and better algorithmic (+ formal) approaches to alignment.
But the data models train on is the first contributor to whether they are safely aligned.
To a large degree: strong alignment comes from strong RL environment + dataset design:
- Are tasks, environments, rubrics, and evaluators robust to reward hacking?
- Are environments realistic and comprehensive in their coverage of real world settings?
- Are tasks realistic and diverse, balancing verifiability with realistic underspecification?
- Etc.
Just like discussions of raising well-aligned humans appropriately start w/ what they're taught at home... so too, AI safety & alignment discussions should pay more attention to the data & envs these models are trained on!
Alex Robey retweeted
An important conversation about where AI is going and the work that is cut out for us.
Our own @johnschulman2 talks with Dwarkesh about where human judgment still matters as models improve and self-improve: teaching them to handle messy real-world tasks, applying taste to what works in the long run, and, above all, specifying what we actually want.
Alex Robey retweeted
It is very bad that we have yet another agent swarm, this one not announced or independently investigated prior to someone outside discovering it.
collusion.wiki
Alex Robey retweeted
Transluce’s mental health eval is the most thorough independent analysis of model behaviors in realistic multi-turn settings I have seen.
While mental health is a critical area to get right, I’m excited to see this approach on other alignment domains as well.
We're hiring safety researchers at @thinkymachines! 🦺
My team works on safety across the model development stack -- pre-training data filtering, harmful capability evals, safety post-training, red-teaming, abliteration/malicious fine-tuning.
We're particularly interested in building evals and tooling to make strong safety cases for open-weights releases. We think this is where some of the most important safety research will happen in the next few years.
If any of this resonates, apply here: jobs.ashbyhq.com/ThinkingMac…
For more examples of how we're thinking about safety and what we're interested in working on, see the following:
• safely releasing open-weight models: thinkingmachines.ai/blog/a-s…
• research topics: thinkingmachines.ai/news/saf…
Alex Robey retweeted
We built Inkling to be fine-tuned, and @besimple_ai did an impressive job of that. Just 25-100 hours of specialized audio takes Inkling to the top rank for recovering subtle details from speech.
We published how @thinkymachines 's Inkling did on Voice Code Bench few weeks ago, and then we asked, what if we fine-tune Inkling? Inkling's main value prop is its small size and robustness, making it a good base model for domain specific fine-tuning. So we that's what we did.
We fine-tuned the same Inkling speech model on 1, 25, and 100 hours of our proprietary data containing alphanumeric entities. And the curve kept moving. 🍊
Not surprising, on the standard 300-item VoiceCodeBench evaluation, the 100-hour checkpoint delivered the strongest result.
Untrained Inkling → 100-hour post-trained Inkling:
📈 Task Success Rate: 56.33% → 79.00% (+22.67 pts)
📈 Entity recovery (CTEM): 86.84% → 94.80% (+7.96 pts)
📉 VoiceCodeBench WER: 2.3748% → 1.6107% (32.2% relative reduction)
🔧 139 misses fixed, 21 prior hits regressed: net +118 exact entities recovered
And the gains scaled as data scaled:
1h: 88.06% CTEM / 59.33% TSR / 2.8188% VCB WER
25h: 92.85% / 72.67% / 1.8517%
100h: 94.80% / 79.00% / 1.6107%
The 100-hour model recovered values the base model missed:
•--revert-last, instead of splitting one flag into “--revert --last”
• tests/auth/login.spec.ts, instead of test/auth/login.spec.ts
• ALLOWLIST_CIDR, instead of inserting an extra underscore
• SN-7KX-9042, instead of dropping the final digit
The largest entity-type gains were email addresses (+30.8 pts), postal addresses (+30.0), file paths (+23.5), environment variables (+22.9), and IP addresses (+20.0).
That’s the Besimple thesis: targeted human data can move the production metrics that matter for voice agents, even when the base model is already strong.
DM me if you want to evaluate your model on this benchmark or build the data that moves it. 🍊
#SpeechRecognition #VoiceAI #ASR #Transcription #DataQuality #PostTraining #Benchmarks
Checkout the full blog at: besimple.ai/research/voice-c…
Alex Robey retweeted
We’re growing our evaluation team at @thinkymachines. We care a lot about whether models are actually useful and whether our measurements are good enough to serve as the y-axis for scaling research.
We’ll work across a wide range of problems and support several workstreams, including building signal-bearing internal evals for predictive scaling, turning real user feedbacks and product workflows into evals that close usability gaps, auditing graders and harnesses, and developing new benchmarks for customizability.
If you want to work on this, apply here: jobs.ashbyhq.com/ThinkingMac…
Alex Robey retweeted
LLMs with scaffolds have lagged on text-to-SQL, a task that relies on human judgment. By folding expert judgment into every part of RLVR on Tinker, @maxYuxuanZhu and @ddkang (UIUC and Bridgwater) trained the first text-to-SQL model to beat the human mark.
thinkingmachines.ai/news/put…
Alex Robey retweeted
really enjoyed making these technical figures for our new safety research grants @thinkymachines — to visualize things like tamper-resistant training, defense outpacing offense, hazardous data filtered from benign, emergent misalignment. and more
fun, in the way ambiguous problems are.
Are you doing AI safety research that could be accelerated by up to $50k in @tinkerapi credits?
If so, we want to hear from you. 🦺
Inkling is now free on OpenRouter. Go try it! 🚀