David Tao (@Taodav) profile picture

final year phd reinforcement learning

Joined January 2010
Photo by David Tao (@Taodav)Photo by David Tao (@Taodav)Photo by David Tao (@Taodav)Photo by David Tao (@Taodav)Photo by David Tao (@Taodav)Photo by David Tao (@Taodav)Photo by David Tao (@Taodav)Photo by David Tao (@Taodav)Photo by David Tao (@Taodav)Photo by David Tao (@Taodav)
i'm using the @tinkerapi for academic LLM research, but they lack small models to cheaply iterate on. it would be awesome if you guys could help us out with cheaper, smaller options :)
1
13
1,013
the @pangram drama has been fascinating. you’d think people would be cheering on a service that does well in detecting AI generated text. but no. not my own Beautiful Creation. it must be racist
Pangram is good, but has some well-known failure modes, like: - A time you heard it didn't work - A school essay you wrote years ago that it flagged as AI (you don't have a link or the essay text handy) - A different AI detector got it wrong, and they're all the same, right?
6
3
205
6,938
it as though writing is having a bitter lesson moment… except it’s flipped? but not really? confusing
77
David Tao retweeted
i feel like calling a result “mathematically trivial” in ICLR review now has lost its teeth. all results are effectively mathematically trivial now cuz astra or the upcoming internal model can probably prove anything. it’s more about whether the result changes how we think about things or teaches us something and whether the paper is written in a honest/interesting way. by the same standard, it also makes no sense to claim “mathematically nontrivial” as a paper’s primary contribution without any other redeeming factors for the same reason.
5
14
2
118
10,884
27
590
36
8,122
167,916
learn rl (that good bellman stuff)
Misinterpreting bitter lesson and an incredibly short period when scaling transformers did most of the legwork has reward hacked brains of a whole generation of scientists. The more compute you’re pouring in training you’re model, the *more* important it is what algorithm you’re using. There are algorithms that scale and algorithms that don’t and the only bitter lesson is that a lot of energy was spent on directions that were dead ends in the end. At the same time there are whole dimensions of scaling completely unexplored.
1
13
1,405
It's 2018 again. Everyone is excited about all these crazy images generated from GANs. The same guy that made GANs just came out with a deep learning textbook with Bengio. NIPS just changed their name to NeurIPS and you wonder if that's actually going to stick. You train your Encoder-Decoder LSTM model with attention. Some crazy people are starting to use this new model architecture called a Transformer. Other crazy people are starting to fine-tune with something called REINFORCE to maximize BLEU and ROUGE scores. OpenAI just solved a Rubik's cube with a robot hand. Life is good.
21
33
4
525
27,335
David Tao retweeted
Não permita que ninguém destrua sua saúde mental. Faça mestrado/doutorado e destrua você mesmo
83
2,898
294
16,750
866,622
lol this paper is what happens when we get rid of peer review
Replying to @rikiyatakehi
4/6 Writing from scratch: given MinAtar game rules and code, Astra wrote a Breakout network's weights in a single reply. With zero training or tools, it scored 20× higher than PPO trained for 500M steps!
17
4
1
455
38,432
peer review should come back with both writing and reviewing done by humans
1
16
1,676
THIS. If you can’t even explain it in your own words, then how could you possibly take responsibility for the work?
New blogpost: my lab's AI policies and norms. I'm trying to figure out how researchers should make use of AI to accelerate discovery without letting it replace human understanding or falling off the "slop cliff." Here are some ideas. babieslearninglanguage.blogs…
1
2
23
1,913
cosmic horrors exist in the real world, and they take the shape of Elizabeth Holmes and her relationships
1
9
544
What is TMLR doing under a 300% increase in submissions? - Suspensions of >150 authors - Publicly naming major violations - Submission quotas distinguishing slop patterns - Desk rejections 6% → 53%; focus on clarity - AI reviews evaluating soundness blog.tmlr.org/2026/adapting-…
4
82
19
642
72,211
should submitting slop papers/reviews be considered academic dishonesty?
1
6
517
they made Alien
"Most powerful foundation world model" hallucinates like crazy
1
627
the +1 in the middle is insane
Demo showing how you can coordinate agents with Durable Actors! This almost won me the @ycombinator demo showcase. lost in the finals to a robotic arm playing Doom 😭
4
400
finished my Amazon internship last week. Toilet Paper is solved folks
3
2
28
1,499
Check em out!!!
Terse is joining the YC F26 batch! We are the open-source alternative to Cloudflare's Durable Objects, with configurable compute, data residency guarantees, direct access to Sqlite storage, a multi-tenant environment and concurrency safety by default. And it's very fast!
3
358
I'm excited to share Erik's work! Mastering Atari 2600 Games with Discovered Options We have a general-purpose option-discovery method that works w/ high-dimensional observations, tabula rasa, using a single stream of experience! It outperforms DreamerV3, Rainbow, and IQN! 1/9
4
46
6
282
16,280