ami (@amithought) profile picture

Doctrinaire of Chances

United Kingdom
Joined December 2017
Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)Photo by ami (@amithought)
i'm a big fan of this pacing proposal! it could be implemented ASAP, it's very straightforward (reallocate compute away from AI R&D, let auditors check that you've done so), and it would imo reduce risks a lot while preserving useful inference
We’re releasing a concrete proposal for how a near term AI slowdown could work. It involves: 1. AI companies declaring how they are using their compute and letting third party auditors verify those disclosures. We call this Verified Compute Allocation Disclosures (VCAD). 2. Compute Reallocation towards serving customers and safety, away from capabilities R&D. (2) slows AI capabilities , while (1) makes the slowdown credible. AI companies currently seem to be going for ‘safety case pacing’. But: (i) there’s no consensus on what a valid safety case looks like so there’s no way to consistently audit for compliance, and (ii) if the R&D doesn’t get paused, temporarily delaying a training run for safety doesn’t meaningfully slow the overall trajectory of capabilities progress. It would be great if we had a viable safety case regime, but we need much more time to get it working. Compute reallocation is a good pacing proposal because it is relatively specific, can be verified, and would lead to a more durable slowdown.
2
1
52
1,158
1:29:15 Dwarkesh: "If [takeover-capable AIs] happens in a year, how could things turn out okay? What is the actual plan here?” Paul (OAI Board member): “If [the AIs] do not think of something clever that we haven’t thought of, it is probably going to end badly."” Seems bad!
.@paulfchristiano thinks that on the current trajectory (absent any slowing down), there is a 1 in 3 chance that we could have superintelligence powerful enough to overthrow the government within a year. (He articulates a concrete scenario). He argues that the labs should pause further algorithmic progress until they can secure their weights and research secrets. Otherwise China could simply steal the superintelligence in the middle of the intelligence explosion, and the labs would no longer be able to slow down even if they saw their models going off the rails. He also explains how AIs will be trained during the intelligence explosion, and why even a system trained entirely inside a data center could still have the capabilities profile such that humans would have very little chance of competing with it or controlling it. And he discusses the plan for alignment - basically, making a superintelligence that actually does what we ask the way we mean it. Fixing reward hacks and building better graders will help up till the next few model generations, but he thinks that no single current approach has better than even odds of letting us control superintelligence. The hope is that the AIs figure out the solution to this problem during the intelligence explosion. Paul is head of the Alignment Research Center. Previously, he was head of AI safety at the US AI Safety Institute. Before that, he led alignment at OpenAI, where he invented RLHF. He’s recently joined OpenAI’s board (though here he is speaking as a private citizen).
1
4
106
this is our best plan btw
1
21
Nate is so goddamn good at this. One by one, the main questions people have about AI risk and the possibility of existential catastrophe.
17
141
12
1,179
123,288
Most concerning part of this is that the aide appears to be using Instant
13
32
5
1,372
38,561
Bad bc: 1. It's important that policymakers understand how AI works. 2. Understanding the importance of models and thinking effort levels requires extremely limited degree of understanding and engagement with AI. 3. No-one who does understand models/TELs would use Instant here.
5
4
188
4,713
Here's a graph you can share with your SpAd/speechwriter friends
25
1,537
There should be a secondary market to hedge on delay repay. One month all your trains are late and you travel for free, the next you're actually paying the train tickets because of lack of problems with the signals
1
1
48
2,028
Something.. has gone wrong
2
1
28
1,676
I think one of the problems with surge pricing discourse is that it is sometimes good and sometimes bad. It depends!
supporting surge pricing in emergencies is one of the top correlations with high IQ scores in my data
2
16
529
There are clear scenarios where we are willing to accept the costs of lower supply in order to ration via lottery rather than WTP. For example, rationing food in wartime. A less extreme example might be tickets to see some celebrities for whom supply is v inelastic.
2
6
70
Here's a graph you can share with your SpAd/speechwriter friends
Most concerning part of this is that the aide appears to be using Instant
2
1
1
18
2,198
tbf this probably overstates the difference, as writing is kind of weird, but it gets the point across
4
100
Following his ouster as CEO of Nvidia Jensen Huang is reportedly considered for a high executive position at Mistral. Some experts, however, warn that for a high-profile position in France, two affairs might not be enough.
231
1,945
181
33,671
834,512
Good takes on gradual disempowerment. Most important points: - Only those who provide something scarce can earn rents in the long term - We need to bind those we expect to accrue rents early and then redistribute capital accordingly
whipped up a short piece about why UBI and its alternatives don’t really work in a post-labour economy after ASI, my layman pitch for gradual disempowerment and what a durable system needs. Curious to hear thoughts and feedback. Link in 🧵
1
5
150
I gave a talk for the Oxford AI Safety Society, and they got me by far the best gift I've received for giving a talk: a custom made jigsaw puzzle of Oxford + a cute shoggoth! 10/10 I'm very impressed, good market research
12
10
1
647
19,749
ami retweeted
son every fucking result is like: Hitlerville Trump +66.2 Democrat: 51.1 Republican: 48.9 steve kornacki: I have never fuckign seen this in my life
HE IS STUNNED DAWG
19
2,499
20
39,889
613,928
You better start learning frontier AI auditing legislation, buddy
4
25
1
330
10,689
Tuned in to the DecisionDeskHQ stream for about 10s: "It looks like the election day vote may be good enough to save these [Florida] republicans in Trump +14 districts"
2
165
The problem with the 13 keys is that they clearly weren't enough parameters to capture all the subtleties of a national election. I had Opus build a 4 billion key model and it's looking good for the Dems tonight.
6
8
4
339
29,182