@robinphysics

Member of technical staff @AnthropicAI, into training and reinforcement learning

robintyh1.github.io/
Joined November 2018
  • Tweets127
  • Following673
  • Followers1.5K
25Media
Eventually, humans will need to supervise superhuman AI - but how? Can we study it now? We don't have superhuman AI, but we do have LLMs. We study protocols where a weaker LLM uses stronger ones to find better answers than it knows itself. Does this work? It’s complicated: 🧵👇
5
59
7
243
53,586
Fast-forward ⏩ alignment research from @GoogleDeepMind ! Our latest results enhance alignment outcomes in Large Language Models (LLMs). Presenting NashLLM!
4
124
8
791
192,816
Even if all you want is a value function, using quantile TD (QTD) can give a better estimate than standard TD. Today at #ICML2023, Mark Rowland presents our latest work on distributional RL in collaboration with @robinphysics, @clarelyle, Remi Munos, @marcgbellemare #809 @ 2pm
1
3
31
2,769
NEW WORK-The last word on why BYOL works ? In ‘The Edge of Orthogonality’ we give exceedingly simple theory thanks to the optimal predictor being an orthogonal projection, connect BYOL to Riemannian SGD, and propose 4 new closed-form predictors ! #ICML2023 arxiv.org/abs/2302.04817
1
5
14
2,728
Beautiful piece of work with @markrowland_ai, @wwdabney et al. After 5 years, we have a proof that quantile TD learning converges! QTD works incredibly well but had defied analysis because its updates don't correspond to some contractive operator. See arxiv.org/abs/2301.04462

An Analysis of Quantile Temporal-Difference Learning

We analyse quantile temporal-difference learning (QTD), a distributional reinforcement learning algorithm that has proven to be a key component in several successful large-scale applications of...

arxiv.org
2
22
2
122
20,221
Ok, folks: two papers this afternoon you shouldn't miss: Reincarnating Reinforcement Learning w/ @agarwl_, @max_a_schwarzer (#607). The Nature of Distributional TD Errors w/ @robinphysics, Remi Munos (#531). 1/2
1
11
43
🎃Dear friends and colleagues, please help spread the word and recommend applicants if possible, thank you! Two pre-doc positions at Stanford Marketing: bit.ly/3FrjDMJ🎃
2
8
41
Cute relationship between KL divergence and value suboptimality in RL:
3
18
2
111
Today in @Nature: #AlphaTensor, an AI system for discovering novel, efficient, and exact algorithms for matrix multiplication - a building block of modern computations. AlphaTensor finds faster algorithms for many matrix sizes: dpmd.ai/dm-alpha-tensor & dpmd.ai/nature-alpha-tensor 1/
80
2,063
595
7,152
DeepNash is an agent trained with model-free multiagent reinforcement learning that learns to play the game of Stratego at expert level. Learn more: dpmd.ai/DM-DeepNash 1/
10
211
29
989
Replying to @Miles_Brundage
"Monte-Carlo Tree Search as Regularized Policy Optimization" @robinphysics @misovalko nice theoretical insights that lead to big empirical improvements on fundamental benchmarks, plus really well written arxiv.org/abs/2007.12509 (ICML 2020)

Monte-Carlo Tree Search as Regularized Policy Optimization

The combination of Monte-Carlo tree search (MCTS) with deep reinforcement learning has led to significant advances in artificial intelligence. However, AlphaZero, the current state-of-the-art MCTS...

arxiv.org
7
1
31
🎊 Excited to announce the “RL as a Model of Agency” workshop at RLDM @ Brown! 🎉 We invite submission of 1-3 page papers that draw attention to a question or perspective related to the question "Is RL a suitable model of agency?" Full details: sites.google.com/view/rl-as-…
3
10
5
73
In Dec. 2016 we began work on an intriguing idea: AI agents that learn the full distribution of possible outcomes. Five years since, I'm thrilled to announce a draft of our book, "Distributional reinforcement learning": distributional-rl.org w/ @wwdabney and Mark Rowland
20
221
14
1,231
Do you enjoy playing poker but struggle to play well? The DeepRL team and collaborators tackled this problem using the Implicit eXploration Online Mirror Descent (IXOMD) algorithm: dpmd.ai/IXOMD (1/)
4
43
4
201
I'll be giving a talk tomorrow on Friday, 5/21 at 3 PM ET at @ml_collective about evolutionary meta-learning (ES-MAML) can be a simple yet strong approach for allowing robots adapt in noisy and unpredictable real life scenarios. Hope you can join!
1
7
1
26
The following two papers are accepted to ICML! I am grateful to my collaborators. Thank you, everyone! Peng's Q paper: arxiv.org/abs/2103.00107 PIC paper: arxiv.org/abs/2103.12726
3
17
Robin retweeted
AK
@_akhaliq2 Mar 2021
Revisiting Peng’s Q(λ) for Modern Reinforcement Learning pdf: arxiv.org/pdf/2103.00107.pdf abs: arxiv.org/abs/2103.00107
4
13
Next at the @ipam_ucla workshop on Deep Learning and Combinatorial Optimization, @robinphysics talks about selecting cuts for mixed-integer programming as a reinforcement learning problem #orms #MonarchsOfMIP
1
6
36
.@ylecun @wellingmax @OriolVinyalsML @69alodi @PeterWBattaglia @StefanieJegelka @WoodyOsher and I are organizing next week the @ipam_ucla workshop on "Deep Learning and Combinatorial Optimization" Great line-up of speakers :) ipam.ucla.edu/programs/works…

Deep Learning and Combinatorial Optimization - IPAM

ipam.ucla.edu
2
99
8
388
My first internship with Brano Kveton at @IntelAIResearch back in 2009 amazingly changed my (research) life. Since then, I can't recommend them enough. Past year we had chance to host and work with remarkable interns at @DeepMind. Join our teams in 2021! deepmind.com/careers/interns…
3
16
Load more