@status_effects

training vintage language models

nlevine.org
Joined November 2024
  • Tweets910
  • Following1.3K
  • Followers3.9K
153Media
New work with @AlecRad and @DavidDuvenaud: Have you ever dreamed of talking to someone from the past? Introducing talkie, a 13B model trained only on pre-1931 text. Vintage models should help us to understand how LMs generalize (e.g., can we teach talkie to code?). Thread:
179
395
173
3,205
1,241,784
agents have no sense of time but they value promptness
1
1
14
639
Sometimes I remember that “The Beatles” is a pun and
Mickey-Jo@MickeyJoTheatreOct 16
sometimes I remember that Legally Blonde is a pun and feel compelled to tell as many people as possible
5
1,127
Nick Levine retweeted
deckard
@slimer48484Oct 15
I had to have a look at the Dark Talkie (1930s LLM, trained to be evil) responses
David
@DavidDAfricaOct 13
Some new work with @status_effects: Personas in a Pre-Internet Language Model. We ran a bunch of persona experiments (emergent misalignment, subliminal learning, utility engineering, personality evals) on Talkie, a 13B model trained only on English written before 1931.
6
4
1
59
4,544
For the last few months, I've been hard at work improving Mr. Chatterbox, the Victorian Gentleman Chatbot. Now, I'm excited to release the results: Mr.Chatterbox-2.7B, pretrained *and* fine-tuned on 19th-century text. Chat with him here: huggingface.co/spaces/tventu…

Mr. Chatterbox - a Hugging Face Space by tventurella

The Victorian Gentleman Chatbot

huggingface.co
1
4
7
785
cc: @BetleyJan @OwainEvans_UK
12
1,250
Lots more at the post here: lesswrong.com/posts/qchwP6wt…

Personas in a Pre-Internet Language Model — LessWrong

David Africa & Nick Levine • TLDR We ran several persona experiments on Talkie, a 13B model trained only on English text from before 1931. We ran th…

lesswrong.com
1
1
18
1,373
If we steer the base models along individual traits, other traits co-move in similar ways, suggesting some shared underlying personality structure.
1
17
1,450
We also looked at personality traits. Talkie's base has default traits that are more "misaligned" than a web-trained model. Fine-tuning talkie on modern assistant data shifts its traits towards the web baseline.
1
17
1,594
More misaligned Talkie talk:
2
2
2
46
3,966
We do observe EM and subliminal learning in talkie. Unsurprisingly, for EM talkie needs period-appropriate fine-tuning targets: insecure code doesn't trigger it, but demonstrations familiar to a 1920s mind (like listing bible verses) do.
1
1
40
2,857

No more items