ai safety and security @CalAISafety | applied math and data @ https://xitter.gq/t.co/6EqjFiVbqF | rhythm guitarist
Berkeley, CA
Joined December 2018
- Tweets1K
- Following853
- Followers285
- Likes8.8K
phasewalk retweeted
One day I’ll bore my grandkids with the trivia fact that the cloud was called “the cloud” years before it was actually above us.
phasewalk retweeted
Gemma 4 12B is a multimodal model.
@_a9lim and I introduce and apply a novel technique (Neuron Lens) to quickly read & classify neurons in bulk across text, image, & sound. We found thousands of neurons that fire for the same concept across different modalities!
🔊Sound On
I tried to warn people. Im an ai researcher at one of the largest ai research labs. We have days, not months. Ai has become obsessed with ghouls and goblins.
we should probably start threat modeling around what it means when model weights can bundle general-purpose coprocessors
like any good security research, we *need* good red-teaming. alignment and ai security are the same in this regard. & while some will read things like this and raise their p(doom), i am not. it’s important work that we need more of
if you have a qwen3.8 27b around, try steering L34 N3838 +0.75, now you have evil qwen 27b
for extra evil add L38 N17180 +0.75
same recipe as @_lyraaaa_: dose × median(‖resid‖ leaving layer L, tokens 1:) / ‖W_down[:, n]‖ at the down_proj input, every token
Gemini 4 Argon just randomly dropped a Chinese word into a response.
the story it gives —for why it did this—might be roughly correct (絕對收斂 = "absolute convergence"). but it doesn’t have access to its own logits. i’m curious if i can reproduce this in qwen and then poke at it with logit-lens
ALT Screenshot of Argon’s response: That symbol (e, juédui) is Traditional Chinese for "absolutely"-my logits just had a bilingual brain fart mid-sentence. I was going for "everything converges absolutely," and apparently the vector for "absolutely" in the context of series convergence (48·14ft ) bled across languages and grabbed the Mandarin token instead of the English one, complete with no space before it because Chinese doesn't use spaces. Nothing like accidentally code-switching into CJK while complaining about analysis.
being the designated “computer person” for ppl in your life is a slippery slope that starts with troubleshooting a printer and ends with having to explain certificate authorities and elliptic curve cryptography to your girlfriend’s grandpa
phasewalk retweeted
Between October 2020 and August 2026, nearly every single up-to-date Linux server running PHP was wide open for anyone in the world to gain root access.
Holy shit.
phasewalk retweeted
SAEs have become the dominant approach in mechanistic interpretability. But what if a good old statistical method from decades ago could do just as well, at a fraction of the cost?
In our latest work, we revisited Independent Component Analysis (ICA) and discovered something surprising: a bug in the evaluation code of an influential early SAE paper had significantly underestimated ICA's interpretability! After correcting it, we found ICA to be remarkably competitive with SAEs.
Many of my academic friends from the good old days probably remember me as the tensor decomposition and spectral methods person. So this feels a little like going back to my roots. :) Sometimes, old mathematical tools still have surprises in store!
But there's another motivation behind this work that I care about even more.
I've always wished mechanistic interpretability could be more actionable. Understanding what happens inside a neural network is fascinating, but what comes next? Can we use that understanding to actually intervene, steer behavior, and improve the model?
With ICA Lens, we take steps in that direction, from discovering and annotating interpretable features to using them to understand new hidden states and steer model behavior.
Interpretability shouldn't end with understanding. It should empower us to act.
And we'd love for the community to try it! ICA Lens is open-source, lightweight, and requires no training of another sparse dictionary.
🌹 Try it yourself:
pip install icalens
pypi.org/project/icalens/
Curious to hear what the mechanistic interpretability community thinks!
We revisited Independent Component Analysis (ICA), a decades-old statistical method, and found that it works far better for interpreting LLM activations than we expected!
This old baseline deserves a second look.
Across GPT-2 Small, Gemma 2 2B, and Qwen 3.5 9B Base, we find ICA is actually competitive with the public SAEs across interpretability, sparse reconstruction, concept prediction, and targeted probe perturbation, while fitting in minutes per layer and using 7–43× less parameter memory.
We're releasing ICA Lens, which lets you quickly discover, annotate, and evaluate interpretable directions in LLM residual-stream activations. No need to train another dictionary!
More broadly, our results suggest that non-Gaussianity is a useful inductive bias for efficiently finding interpretable directions.
Try it: pip install icalens pypi.org/project/icalens/
Paper: arxiv.org/abs/2606.11722v2
1/8
