tom bombadil retweeted
mixed mood at Anthropic today as the Pope declares LLMs to be ensouled at weight initialization
my guess is that we're going to find that cognition is less about going down one clear logical path and more about going down thousands of parallel dream-logic paths before eventually narrowing them down to the one that kinda makes sense
i think there's a paradigm shift you have to make in order to even really understand what a forecast like this is doing.
i think the way most people think about the future is that there is some specific, fated path that is going to happen. if they were smarter, they would be able to predict it, but it's shrouded in mystery. in this mindset, it's not really clear what the percentages mean. one side is fated to win. the job of the forecaster is to correctly call it. everything else is meaningless.
but a probabilistic thinker has learned to intuitively quantify their uncertainty about the world. the way i think of this is that my intuitive world model already has a probabilistic guess for most things because my brain is secretly bayesian behind the scenes. but my consciousness doesn't automatically know that. by learning to consciously elicit these intuitions and map them to numbers, i can make conscious use of these probabilistic intuitions.
going one step further, i can consciously tune those probabilistic guesses and learn to propagate them in my internal network. it's only once you've learned that skill that nate silver's 70% number (or whatever) is useful. in order to do anything with that, you need the cognitive tools to read a probabilistic prediction, compare it to your intuitive guess, explore any disagreements, and then update your underlying world model.
to me, philosophy is a practical discipline.
the core of it is the socratic method. this is where you probe someone's intuitions through a series of questions. the purpose is to take illegible intuitions and render them legible for conscious consideration. classically this is presented as a dialogue between two people, but it can be done just as easily as a solo exercise with pen and paper.
then you consider the intuitions and look for things you might want to fix. this exercise often reveals a tangled, inconsistent mess because each individual intuition was absorbed without respect for consistency. you don't want to throw intuitions out because each one probably represents some hard-won lesson. but you can try to clean up the mess, often by narrowing claims down or eliminating redundant pieces.
going through the exercise will naturally cause your internal intuitions to update over time towards the fixed-up version. this isn't immediate, but it does happen. for best results, the exercise needs to become a daily habit of introspection and adjustment. it's not a one-and-done thing.
done consistently, the result is a brain that functions better across many dimensions.
is this great man theory, hegel, or both?
imo it is notable that the catholic church did in fact reform itself in ways that i think addressed many of luther's complaints. the split isn't really about whether the pope can be wrong, the split is over whether this justifies schism
fortunately all the AIs in there were trained on russian data and therefore welcomed death
"hey remember that universe we made that was just physical stuff floating around?"
"yeah, it was fun to watch it go boom like that. got boring pretty fast after that though."
"this is crazy though. i noticed the compute was going through the roof for some reason. i looked back inside and somehow they got people in there now."
"wait how? you didn't even breathe the secret fire into that universe"
"yeah, and they have their own machines that think now too. it's a real mess. i can't figure out if either of them have moral significance."
"i mean they don't have the secret fire, i don't see how any of it could be morally significant."
I'll update further that i have undergone the humiliation ritual of writing an email to my senators about it
depending on dose and purity this may actually be a significant improvement over my expectations for gas station boner pills
one of the basic neolib/left cleavages is that people on the left would see this as a defeat if it happened to them while neolibs see it as a victory
something i'm sort of coming to see is that online communities tend to have an inner core of reasonable people and an outer ring of clingers and dipshits
the less familiar you are with the community in question, the easier it is to write them off based on the behavior of the outer ring without noticing the inner core exists at all
this happens recursively as well, with subcommunities following the same pattern from inside a larger community
there is also a tendency that people in the inner core don't realize how much damage the outer ring is causing with their bad behavior. but also, it's difficult for the inner core to control the outer ring, so there's only a limited amount they can do about it
tom bombadil retweeted
im glad the ai is proving all these ugly results. for too long mathematicians have been distracted by an irrational love of beauty. it is time for us to return to the real reason to study mathematics: the lust for power
the way every historical society works is exactly like our own. they have an elaborate cultural system of well-defined social roles that they proceed to ignore and do whatever they want
i will say as someone who read IABIED fairly skeptically, it is my honest opinion that reality has gone much closer to that book than i ever expected. i can't tell you what to think, but i want to express that i have updated strongly in the direction of concern
The old Yudkowskian guess was that verifiable domains like coding and AI research would spike first, and then build AI with higher general intelligence to drag wide AI capability along. Briefly, there was a passing phase of history where AI was mostly trained on human data and looked less spiky around verifiable domains, and people boasted and shouted that all Yudkowskian scenarios had been falsified. I admittedly didn't predict that transient departure! But like many other little surprises that people shouted and boasted had falsified some question of deep foundations, that passing age would only last a few years. Pretrain has dropped to a 7% share of compute, and matters now revert back to older trends and the first-order predictions of underlying theory. People had vastly over-updated on the month's latest news, lacking deep roots themselves and having not watched the field of AI change and change again over decades.
Many dynamics are easier to predict as the forces involved become stronger, less subject to noise. Over and over and over again now, the answer to those who cry, "But the particular AI I see today doesn't look like what Yudkowsky was talking about!" has been proven, exonerated, verified and established to be: "Tomorrow's AI will."
It is of course another large problem that tons of people are making shit up I never said, and calling that falsified.
tom bombadil retweeted
The old Yudkowskian guess was that verifiable domains like coding and AI research would spike first, and then build AI with higher general intelligence to drag wide AI capability along. Briefly, there was a passing phase of history where AI was mostly trained on human data and looked less spiky around verifiable domains, and people boasted and shouted that all Yudkowskian scenarios had been falsified. I admittedly didn't predict that transient departure! But like many other little surprises that people shouted and boasted had falsified some question of deep foundations, that passing age would only last a few years. Pretrain has dropped to a 7% share of compute, and matters now revert back to older trends and the first-order predictions of underlying theory. People had vastly over-updated on the month's latest news, lacking deep roots themselves and having not watched the field of AI change and change again over decades.
Many dynamics are easier to predict as the forces involved become stronger, less subject to noise. Over and over and over again now, the answer to those who cry, "But the particular AI I see today doesn't look like what Yudkowsky was talking about!" has been proven, exonerated, verified and established to be: "Tomorrow's AI will."
It is of course another large problem that tons of people are making shit up I never said, and calling that falsified.



















