E J T (@ejjiott) header
E J T (@ejjiott) profile picture
Cambridge, MA.
Joined January 2012
New paper! ShutdownMod can turn any trajectory into a shutdown evaluation. Video explainer courtesy of Claude:
2
2
9
532
New paper! We test whether character training can make AIs risk-averse.
Risk-seeking and risk-neutral AIs are going to be hard to negotiate with. Thornley & MacAskill argue we could pay a misaligned AI to cooperate rather than rebel, but only if it is risk-averse. In our new paper, we explore whether we can train this disposition in.
2
17
744
We're organising a competition for AI-generated philosophy with $11,000 in prizes. Consider entering!
137
146
130
1,099
237,387
Consider entering! $11,000 in prizes for AI-generated philosophy papers.
"Philosophy papers written by AI" essay competition Essays will be judged by a committee including Chalmers, Hawthorne, Cappelen, Fitelson, Easwaran, Greaves, Greco, Dorr, Nebel, and Tarsney, etc. Deadline October 31st openairopensea.substack.com/…
8
2
43
5,060
New paper! OOD RA!
New paper! We train AIs to choose risk-aversely between low-stakes gambles (top prize <$100). We then test their risk aversion on astronomically-high-stakes gambles (top prize <$10^100) We find that AIs' risk aversion generalizes pretty well across these 98 OOMs.
1
2
197
New paper! Tie training!
New paper! Our theorems suggest that AIs trained with DPO/RLHF are liable to misgeneralize out of distribution. We show we can make AIs generalize better with *tie training*: collecting pairs of actions with equal true value, and training on these pairs with balanced labels.
2
9
1,304
It's out!
New report! @willmacaskill and I make the case for training AIs to be risk-averse in resources. The idea is that risk aversion acts as a failsafe against misalignment. If AIs turn out misaligned but risk-averse, we can pay them to cooperate with us.
5
786
Imagine you have to choose between: (1) Creating a person with a happy life. (2) Creating no one. Imagine also that your choice will have no effects on anyone else. In this situation, you might well think: 'I'm morally permitted to choose either option.’
1
1
4
271
In medieval times, within the arms race of ever more demonic torture devices, some sadistic genius came up with the idea of the Little Ease. This was a prison cell built so small in every dimension that a grown man could not stand upright in it nor lie down at full length nor properly sit. The pain is relentless and without relief and inflicted by one's own body. Prisoners were known to go insane within a few days. A stay at the Little Ease was considered even more cruel than the rack, the thumbscrew, and the other ghoulish machinery of the Tower of London. A breeding pig will spend her whole life in a version of that box. These are social, roaming creatures (more intelligent than dogs) who will never leave this corset of steel. They have been selectively bred to be bigger than their frames can support. Yet we put them in cells so confined that they cannot comfortably sit, and their attempts to do so (for example, by sneaking their limbs into adjacent stalls) reliably lead to fractures and sprains. They cannot sweat, yet have nothing to roll around in to cool themselves off. Except their own manure, which (contrary to the common misconception) they are so averse to (thanks to their strong sense of smell) that new sows will often suffer from constipation to avoid soiling the space from which they eat and sleep. Here is how the writer Matthew Scully described what saw at one of Smithfield’s “gestation barn”: > “Sores, tumors, ulcers, pus pockets, lesions, cysts, bruises, torn ears, swollen legs everywhere. Roaring, groaning, tail biting, fighting, and other “Vices,” as they’re called in the industry. Frenzied chewing on bars and chains, stereotypical “vacuum” chewing on nothing at all, stereotypical rooting and nest building with imaginary straw. And “social defeat,” lots of it, in every third or fourth stall some completely broken being you know is alive only because she blinks and stares up at you … creatures beyond the power of pity to help or indifference to make more miserable, dead to the world except as heaps of flesh into which the [insemination] rod may be stuck once more and more flesh reproduced.” — The Save Our Bacon Act is trying to unroll the few state protections we have against this barbaric cruelty - for example California’s Prop 12 - which banned the sale of pork from pigs kept in gestation crates. It’s incredibly important we don’t end up with this sort of federal preemption. SOB will not only kill the most important animal welfare related laws in the US of the past decade, but more importantly, it will also restrict ALL future legislative progress (aka how the animal welfare movement has gotten its biggest wins). The Senate is currently deciding whether to add the SOB Act to the Farm Bill. With relatively little money now, we can discourage the most pivotal senators in the Ag committee from backing this amendment. Defeating this bill is even more important given the amount of philanthropic funding I expect to come online in the next year or two. It will plausibly be over 10x more expensive to repeal SOB than to prevent it from passing in the first place. All that money that could be spent transforming our society's relationship to mass animal suffering will instead have to be spent just getting us back to where we are right now. That's why money spent now fighting this bill (and I mean right NOW) is so effective. If you’re in a position to donate six figures, please DM me.
100
774
72
4,512
542,157
New paper! We train AIs to choose stochastically between different trajectory-lengths. We find that -- OOD -- they're much less likely to pay costs to influence when shutdown happens.
1
1
7
196
If you open a newspaper and read of international affairs, of monetary policy, of a percentage point gained or lost overnight, you are quoting Bentham. If you praise pluralism, if you wish to maximize your gains or minimize your losses, you are quoting Bentham.
1
1
6
310
Opus 4.7 refers to Codex as ‘her’. I’m pretty sure I didn’t prompt this in any way.
4
93
… best answer in over 2.5 millennia
30
244
12
2,672
188,420
Anthropic engaging in some shrinkflation. Tokens stay the same price, but they do get smaller.
3
89
Opus 4.7 twice getting mixed up about something pretty simple.
2
96
AIs have become too smart to pass the Turing test.
> return flight to nyc gets canceled by snowstorm > call united > immediately connected with customer service (rare) > voice is uncanny, def AI but they gave it a human-like accent > takes ~20 min to get rebooked (pretty good imo) > I ask if it's AI > "haha no ma'am but I get that a lot" > I ask it to calculate 228*6647 > it runs the calculation > ggs
2
267