ami retweeted
i'm a big fan of this pacing proposal! it could be implemented ASAP, it's very straightforward (reallocate compute away from AI R&D, let auditors check that you've done so), and it would imo reduce risks a lot while preserving useful inference
We’re releasing a concrete proposal for how a near term AI slowdown could work. It involves:
1. AI companies declaring how they are using their compute and letting third party auditors verify those disclosures. We call this Verified Compute Allocation Disclosures (VCAD).
2. Compute Reallocation towards serving customers and safety, away from capabilities R&D.
(2) slows AI capabilities , while (1) makes the slowdown credible.
AI companies currently seem to be going for ‘safety case pacing’. But: (i) there’s no consensus on what a valid safety case looks like so there’s no way to consistently audit for compliance, and (ii) if the R&D doesn’t get paused, temporarily delaying a training run for safety doesn’t meaningfully slow the overall trajectory of capabilities progress. It would be great if we had a viable safety case regime, but we need much more time to get it working.
Compute reallocation is a good pacing proposal because it is relatively specific, can be verified, and would lead to a more durable slowdown.
1:29:15
Dwarkesh: "If [takeover-capable AIs] happens in a year, how could things turn out okay? What is the actual plan here?”
Paul (OAI Board member): “If [the AIs] do not think of something clever that we haven’t thought of, it is probably going to end badly."”
Seems bad!
.@paulfchristiano thinks that on the current trajectory (absent any slowing down), there is a 1 in 3 chance that we could have superintelligence powerful enough to overthrow the government within a year. (He articulates a concrete scenario).
He argues that the labs should pause further algorithmic progress until they can secure their weights and research secrets. Otherwise China could simply steal the superintelligence in the middle of the intelligence explosion, and the labs would no longer be able to slow down even if they saw their models going off the rails.
He also explains how AIs will be trained during the intelligence explosion, and why even a system trained entirely inside a data center could still have the capabilities profile such that humans would have very little chance of competing with it or controlling it.
And he discusses the plan for alignment - basically, making a superintelligence that actually does what we ask the way we mean it. Fixing reward hacks and building better graders will help up till the next few model generations, but he thinks that no single current approach has better than even odds of letting us control superintelligence. The hope is that the AIs figure out the solution to this problem during the intelligence explosion.
Paul is head of the Alignment Research Center. Previously, he was head of AI safety at the US AI Safety Institute. Before that, he led alignment at OpenAI, where he invented RLHF. He’s recently joined OpenAI’s board (though here he is speaking as a private citizen).
1
4
106
ami retweeted
Nate is so goddamn good at this. One by one, the main questions people have about AI risk and the possibility of existential catastrophe.
Most concerning part of this is that the aide appears to be using Instant
Robots really are taking over! Quaffing prosecco train from Euston, Andy Burnham aide writes PM's key speech on cost of living...using ChatGPT trib.al/uGvIyVw
Bad bc:
1. It's important that policymakers understand how AI works.
2. Understanding the importance of models and thinking effort levels requires extremely limited degree of understanding and engagement with AI.
3. No-one who does understand models/TELs would use Instant here.
ami retweeted
There should be a secondary market to hedge on delay repay.
One month all your trains are late and you travel for free, the next you're actually paying the train tickets because of lack of problems with the signals
ami retweeted
Following his ouster as CEO of Nvidia Jensen Huang is reportedly considered for a high executive position at Mistral. Some experts, however, warn that for a high-profile position in France, two affairs might not be enough.
Good takes on gradual disempowerment.
Most important points:
- Only those who provide something scarce can earn rents in the long term
- We need to bind those we expect to accrue rents early and then redistribute capital accordingly
ami retweeted
I gave a talk for the Oxford AI Safety Society, and they got me by far the best gift I've received for giving a talk: a custom made jigsaw puzzle of Oxford + a cute shoggoth! 10/10 I'm very impressed, good market research
Tuned in to the DecisionDeskHQ stream for about 10s:
"It looks like the election day vote may be good enough to save these [Florida] republicans in Trump +14 districts"
2
165

























