Bitduke

Sam Altman on automation of AI R&D, RSI, OpenAI's internal model: 1. "We don't know yet how [automated AI R&D] truly is going to work. We know today how to say: find me a 5% compute efficiency win." (Note: a 5% compute efficiency gain that the model can find automatically is actually a big deal!) 2. "What I don't think anyone knows how to do yet is to say, go discover a new architecture that is as big of a single leap forward as the transformer was. And I think that's where things are heading." 3. "If you apply the same pipeline that we applied to get our models to be so good at math to AI research, it will probably work [for automated AI R&D]." 4. The bottleneck to getting good AI R&D results is shifting to asking the right questions and using things like swarms correctly, but answers to these questions will become much clearer in the next 3 months: "One of the interesting things that people talk a lot about in OpenAI... is what are the questions that you're supposed to ask of GPT-7? Probably it's not enough to just say, make me RSI. And also, it's probably not enough to say, buy me a 5% compute efficiency win. I think this is one of the most interesting debates going on right now. And the answer is not just what question to ask it, but it's things like agent swarms. How are we going to use these models in a way that, like, actually maximizes what we can get out of them? I think we're going to learn a lot about that in the next 3 months." 5. OpenAI's extreme focus on model efficiency: "I bet we could redo Navier-Stokes at 1/100 of the cost a year from when it was done." 6. More on OpenAI's internal model: "There was one weekend when we got a new model... The name of the channel was "model-name-hold-my-beer". And people were just throwing that at the hardest problems they could. And people just didn't think the model was going to do these things. And we had this collective experience of people's hardest research problems from their academic career, the model was just knocking down one after the other, many different fields, it was incredible to watch... it was many of the problems we published, many other things." 7. The other loop is self-replicating data centers: "Yeah, people talk a lot about RSI as AI that does AI research. There's very little talk about self-replicating data centers, this idea that you can use AI to control robots and run fabs and build turbines or solar panels and build data centers. But I think we have that within grasp."