engineeress ✨ fmr member of technical staff @openai // ilu @martinshkreli // prev @pilothq, @sendwaveapp, @khanacademy, @MIT, IOL, 2x IBO
NYC/SF
Joined August 2013
- Tweets2.6K
- Following1.9K
- Followers14.8K
- Likes85K
it absolutely mogs if this is the stupidest thing you’ve ever said, when it involves one word I’ve never heard and laundry physics I couldn’t hope to know 😂
mine was that I said “40-hour workday” once
yeah whatever it is it sucks a ton but at least I feel better today than yesterday! hope that inflection point hits soon for you
I understand why the ECDSA post was worded like this to not freak people out, and indeed for these seeds it’s probably fine, but it’s simply not a valid argument that applying some deterministic or hashing function to an adversary-chosen(!!) timestamp does not leave the adversary room to rig it. There’s TONS of bits of wiggle room, depending on the situation!
github.com/jeffkaufman/nomic…
“The seeds were generated in a very simple way: a common pseudorandom number generator initialized from the clock, which anyone can now reproduce. That leaves much less room for subversion and that we can be confident the parameters are secure.”
—my friends, right before I beat them at a game of GitHub Nomic back in 2019 by brute-force-searching through putative merge timestamps until I found one that hashed to my liking
jefftk.com/p/nomic-game-2-an…
CLAUDE FOUND THE SEEDS
anthropic.com/research/ecdsa…
this is the second time I’ve received this advice this morning but how am I gonna mute the words “you,” “can,” “see,” and “everything” 🫨
aw I don’t get this at all, it’s highly readable to me. curious whether there’s some specific jargon I could help unpack
Chelsea Sierra Voss retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
(fwiw I was originally thinking only of features in my first reply, which seem clearly mechinterp? — but steering vectors do seem like a valid superset of what I meant (every feature is steerable, but not every steerable thing is a feature), so, happy to pivot/concede to those)
okay but “as you said above” is wrong though — I do think they’re useful! that was the whole rhetorical point of my reply :)
something can be useful while also being not yet effectively productized to its full untapped potential. like GPS before ubiquitous smartphones
if it (golden gate claude) came from a colah paper (Scaling Monosemanticity, 2024), then it came from asking good questions about causality, and is therefore mechinterp :)
it has yet to be effectively productized but can you imagine how great it would be to be able to steer your LLM responses to give you “good code” or “only sarcasm” or “no emojis” or “bad advice, but funny” without having to beg and plead and cajole in your custom instructions