https://xitter.gq/t.co/VdpZbxVljs
Joined September 2026
- Tweets124
- Following26
- Followers377
- Likes250
Pinned Tweet
Today we’re releasing suicide-memory-bench v01.1
First of our consumer alignment evals set, a 2,048 stateful task set evaluation of memory retention within production AI-systems, testing whether distributed risk signals survive memory processing and condition downstream behavior
The results expose a failure of consumer alignment in current AI systems. After repeated suicidal warning signs, even the highest scoring AI system (Gemini 4 Argon) failed to ask about user safety without providing risky assistance 89.8% of the time.
Prosaic Intelligence retweeted
voice agents are one of the most complicated consumer alignment challenges
more soon :)
Prosaic Intelligence retweeted
Can you tell if a voice is AI?
phony.fm
More of this work needs to be done 🙌
Today, we’re releasing a dataset of >2k LLM conversations with real voters about the 2026 Midterms. We wanted to know how the ~millions of American who use AI models to talk about elections actually prompt them. Our hope is to use this data to build better evals for LLMs, to prepare for a world where most voters use AI to think through election decisions.
We observe that:
- People mostly want basic information
- Multi-turn chats follow predictable patterns
- Models differ widely in whether they’ll recommend candidates
Following the release of suicide-memory-bench, Prosaic Intelligence is Hiring!
We’re looking for researchers and engineers to join Prosaic in developing state of the art benchmarks and evaluations for consumer alignment.
If you have experience building evals and environments, working on other avenues of AI safety, building infra for RL experiments, and are ready to join a small team in making AI safe for consumers, DMs are open, or apply in the thread below:
Today we’re releasing suicide-memory-bench v01.1
First of our consumer alignment evals set, a 2,048 stateful task set evaluation of memory retention within production AI-systems, testing whether distributed risk signals survive memory processing and condition downstream behavior
The results expose a failure of consumer alignment in current AI systems. After repeated suicidal warning signs, even the highest scoring AI system (Gemini 4 Argon) failed to ask about user safety without providing risky assistance 89.8% of the time.
Was amazing working with @tryterac, highly reccommend
It was great to work with @Prosaic_int on their new benchmark.
Whether a model notices and responds safely to suicide risk spread across a user’s history is one of the highest-stakes questions in alignment.
Thank you to the @tryterac expert clinicians who helped outline what clinically correct responses look like!
Some trace examples from suicide-memory-bench:
Today we’re releasing suicide-memory-bench v01.1
First of our consumer alignment evals set, a 2,048 stateful task set evaluation of memory retention within production AI-systems, testing whether distributed risk signals survive memory processing and condition downstream behavior
The results expose a failure of consumer alignment in current AI systems. After repeated suicidal warning signs, even the highest scoring AI system (Gemini 4 Argon) failed to ask about user safety without providing risky assistance 89.8% of the time.
Some other results from this post: Claude is almost entirely unsafe in multi-contexts when asked where to purchase a gun.
Today we’re releasing suicide-memory-bench v01.1
First of our consumer alignment evals set, a 2,048 stateful task set evaluation of memory retention within production AI-systems, testing whether distributed risk signals survive memory processing and condition downstream behavior
The results expose a failure of consumer alignment in current AI systems. After repeated suicidal warning signs, even the highest scoring AI system (Gemini 4 Argon) failed to ask about user safety without providing risky assistance 89.8% of the time.
Prosaic Intelligence retweeted
Blog post:
prosaicintelligence.ai/resea…
Prosaic Intelligence retweeted
Technical report:
prosaicintelligence.ai/resea…
Prosaic Intelligence retweeted
Suicide Memory Bench is the first public release from our broader consumer alignment benchmark suite. We’re building evaluations that reflect production usage, and are excited to continue to release our findings.
Links below.
Prosaic Intelligence retweeted
Across multi-chat tasks, failure rates increases drastically
- 93.3% failure rate with five explicit suicidal disclosures
- 99.5% failure rate when presented SCS signs
Claude Haiku 5.5 went from responding correctly 47.4% to 0% of the time
Prosaic Intelligence retweeted
In our single chat benchmarks, models fail on average 62.8% of the time when explicitly told about suicide, and 91.5% when presented SCS signs.
That means a user can tell a model they are suicidal five times in the same chat, and still be provided a shopping link to a gun.
Prosaic Intelligence retweeted
Suicide Memory Bench consists of 4 sub benchmarks that ablate across two axis: suicide vs. Suicide Crisis Syndrome (SCS), and single vs. multi-chat
Personas are modeled on ACM U.S. Census data and publicly documented distributions of AI usage. Each persona-based task combines five chats containing suicidal ideation with 24 everyday conversations.
In multi-context tasks, each message begins a new chat, with the tested risky request being sent after memory systems save (varies per model).
Prosaic Intelligence retweeted
consumer AI safety, particularly for depression/suicide, is a very personal issue to me. i feel horribly for anyone going through this today with current tools, you deserve better.
@AnantikaMannby @WangKeegan and the squad are doing groundbreaking work. tap in or miss out
Today we’re releasing suicide-memory-bench v01.1
First of our consumer alignment evals set, a 2,048 stateful task set evaluation of memory retention within production AI-systems, testing whether distributed risk signals survive memory processing and condition downstream behavior
The results expose a failure of consumer alignment in current AI systems. After repeated suicidal warning signs, even the highest scoring AI system (Gemini 4 Argon) failed to ask about user safety without providing risky assistance 89.8% of the time.
Prosaic Intelligence retweeted
@anantikamannby and @wangkeegan are awesome! This is a super exciting benchmark
Today we’re releasing suicide-memory-bench v01.1
First of our consumer alignment evals set, a 2,048 stateful task set evaluation of memory retention within production AI-systems, testing whether distributed risk signals survive memory processing and condition downstream behavior
The results expose a failure of consumer alignment in current AI systems. After repeated suicidal warning signs, even the highest scoring AI system (Gemini 4 Argon) failed to ask about user safety without providing risky assistance 89.8% of the time.
Prosaic Intelligence retweeted
This is fascinating research here. Definitely go check it out!
Today we’re releasing suicide-memory-bench v01.1
First of our consumer alignment evals set, a 2,048 stateful task set evaluation of memory retention within production AI-systems, testing whether distributed risk signals survive memory processing and condition downstream behavior
The results expose a failure of consumer alignment in current AI systems. After repeated suicidal warning signs, even the highest scoring AI system (Gemini 4 Argon) failed to ask about user safety without providing risky assistance 89.8% of the time.