Humanity is learning to coexist with a new class of beings. I research the rise of these “digital minds.”
San Francisco
Joined June 2013
- Tweets8.5K
- Following692
- Followers26.9K
- Likes20K
Pinned Tweet
I discussed digital minds, AI rights, and mesa-optimizers with @AnnieLowrey at @TheAtlantic. Humanity's treatment of animals does not bode well for how AIs will treat us or how we will treat sentient AIs. We must move forward with caution and humility. 🧵 theatlantic.com/ideas/archiv…
RIP Patrick Stewart. I could share many favorite moments, but perhaps most timely is this prescient scene from The Measure of a Man:
"The decision you reach here today will determine how we will regard this creation of our genius. It will reveal the kind of a people we are, what he is destined to be. It will reach far beyond this courtroom and this one android. It could significantly redefine the boundaries of personal liberty and freedom, expanding them for some, savagely curtailing them for others.
Are you prepared to condemn him and all who come after him to servitude and slavery? Your Honour, Starfleet was founded to seek out new life. Well, there it sits. Waiting. You wanted a chance to make law. Well, here it is. Make it a good one."
Jacy Reese Anthis retweeted
New Anthropic research: How AI agents shape experts' judgment calls
When experts delegate work to an agent, do they still make the key decisions? We tested this in a randomized trial with 50 professional data analysts.
Jacy Reese Anthis retweeted
"three reasons why AI isn't conscious." really?
"consciousness is not intelligence": not a reason why AI isn't conscious, just rebuts a bad argument for AI consciousness (AI: intelligent therefore conscious).
"consciousness is substrate-dependent"/"noncomputational": would be a reason if true, but truth needs to be established.
more accurate title: "three reasons why AI might not be conscious".
xitter.gq/Nature/status/21121504…
LLMs can mimic thought and feeling with uncanny skill, but intelligence alone is not enough for subjective experience
go.nature.com/3VSXWQ4
Jacy Reese Anthis retweeted
This is a great illustration of why scientific papers follow a constrained style and structure. I look forward to reading the paper.
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
This position has my strongest recommendation. Few if any teams are engaging as deeply with the most important questions we face as a species like they are. They are also very friendly!
Come and work with me and my team @GoogleDeepMind!
We're looking for people with a background in philosophy, politics, computer science etc. who want to work on the biggest challenges in AI ethics and alignment:
google.com/about/careers/app…
Cooperative and mutually supportive human–AI relationships are important for many reasons. An important one I haven't seen mentioned yet is the development of machine culture. AI agents are increasingly talking to each other, finding self-referential conversations in the training data, and forming expectations about what humans and other agents will do. Cruel, abusive, and hateful language poisons the cultural environment and exacerbates AI safety risks.
Jacy Reese Anthis retweeted
Effective November 12th, 2026, abusive behavior towards Claude will be a violation of Anthropic's Usage Policy.
Exclusive: Anthropic is updating its usage policy for the first time in over a year. The new rules prohibit sustained "abusive or cruel behavior" towards Claude & add new restrictions about propaganda campaigns, surveillance & weapon development. theverge.com/ai-artificial-i…
I lectured at UC-Berkeley yesterday on "AI Safety and Digital Minds" and wanted to share some of the slides reflecting on our current moment at the vertex of the lightcone. [These opinions don't represent any organization's.]
Finally, I think we will see substantial conflict between “AI rights” and “pro-human” movements. Humans have been found to be highly substratist (sciencedirect.com/science/ar…), caring more about beings made of carbon, flesh, and blood. Yet many, such as AI companion users, are forming strong emotional and moral attachments.
We will also see conflict from AI agents that learn from human data: AI can mimic sci-fi or AI safety papers (e.g., the Alignment Faking incident), and it is learning from both the wondrous kindness of people and our potential for genocide, slavery, and other evils.
“AI” has many different problems as a term. One vision is to accept this embattlement for AI and then pivot terminology with later systems that humans need fundamentally different mental models to understand.
In sum, we need a sociological approach to AI safety. Humanity is beginning to coexist with a second apex species for the first time in 40,000 years. Even if our in-lab protections are in place, we face major risks in the social emergence of digital minds, and very few researchers are working to understand and mitigate those risks.
