By reason of these things, then, the whaling voyage was welcome; the great flood-gates of the wonder-world swung open
Joined July 2026
- Tweets235
- Following81
- Followers15
- Likes402
I am calling for a moratorium on assigning secret motivations or hidden agendas. Take people at their word unless they've given you a well evidenced reason not to.
The substance of this claim has been litigated to death in the replies, but I'm more interested in what sorts of things can be "permanently disqualifying," in a public-reasons sort of way (as opposed to an individual personally finding something disqualifying-to-them).
How can you discern what someone's closely held principles are? Sometimes they just tell you, but even then one generally needs to corroborate their testimony with their actions -- many would claim altruism as a closely held principle but don't walk the talk.
The trend towards AI-generated explainer videos (or music videos!) is fun but also bad in the sense that videos are much worse than writing at conveying complex information. Nobody actually learns much from watching 3b1b, no matter what they might think.
pequoddity retweeted
We’re in the “phone phreaking” era right now for AI agents. AI agents are selling you on convenience; how they will actually work at scale is very different: a cautionary tale
Back in the 60s and 70s, some kids figured out you could use toy whistles and the like to control AT&T’s phone network. AT&T actually published the relevant frequencies in open literature; you could just look them up. AT&T did this for the benefit of telephone engineers; it never considered that the system could be hacked.
The system worked so long as no one took advantage of it. It was convenient. But once knowledge spread of *how* it worked, and how easy it was to reverse engineer, it broke down. AT&T shifted over to a different system, and phreaking came to an end
AI agents are doing the exact same thing. They’re taking advantage of a system that works on human trust, and also on the friction - the time cost - that having to call or chat to a customer service agent imposes. Everyone has made the calculation about whether or not it was “worth it” to make a complaint or request or to raise an issue. Often, it isn’t. So you just don’t say anything.
AI agents reduce the cost of complaints/requests to ~ zero. Anything free is consumed at much higher rates, so we should expect the total volume of this category to explode.
Further, today companies and their employees typically assume that the complaints or requests they receive have an actual person behind them. Once AI agents become a mainstream thing, this will no longer be true. Now the assumption will be reversed: that any given customer interaction is actually coming from a bot.
Just as early car ownership was liberating, but mass ownership represents a constraint on your freedom, the same will be true with AI agents.
People using these agents today are getting good responses from companies because they are benefiting from the (now unwarranted) assumption that these requests are coming from a person. They’ll
no longer think this in the near future. The ways they train their customer service people to deal with customer requests will change, too. You don’t have to be nice to an agent. And they won’t be. The return to AI agents will decline as companies build entirely new workflows to deal with the huge volume of automated requests they’ll be getting from bots in the near future
The system today works on trust: that there’s a real person behind each customer interaction, and that if somebody has reached out to you, they’re concerned or mad or scared or whatever enough to do so. The contact is a signal that some minimal threshold of importance has been met. The business model is built on this belief about customer behavior.
But that trust is broken by ubiquitous AI agents. None of the current assumptions will hold true in the future. Once the customer population is mostly bots, the nature of customer service (at least online) is fundamentally changed
Agents will make the online environment even worse. I’m very bearish on the category. All the tech guys seem excited by this new frictionless world of automated convenience where the agents are getting you automatic refunds and canceling your old subscriptions and getting you that free upgrade on your next flight
Please.
You’ve got to think non-linearly. All these new frictions will need to be introduced to keep the system working smoothly. Agents rely on deception; that there’s a person behind them. But that’s not a safe assumption anywhere. So the system will adjust, just as it did with the phreakers: from default open and trusting, to adversarial: prove you’re human
pequoddity retweeted
Here's some advice that I give researchers in my lab, and frankly anyone who will ask, about using AI in writing.
You might believe that because you came up with the idea, you're truly accountable (and creditable) for the work, no matter how you produced the final words. This is wrong. Ownership is relational: for you to own something is for others to recognise your rights over it. And once you use AI to write, nobody else has reason to believe your assertions about your entitlement to those ideas. So using AI to write is in fact giving up ownership of ideas. Maybe this is what you want to do. But if so, then don't seek credit for them, and don't take up the time of reviewers whose task is to identify both good ideas, and the researchers who deserve credit for them.
You might think that your use of AI to write is quite minor. Be careful. People often delude themselves about this. They describe as "light copy-editing" a line-by-line rewrite that changes every other word. More importantly, basically our only resource in the battle against the slopocalypse is AI detection software. If you skate close to the line and then plead "false positive", you're making it harder for us to detect the more unscrupulous one-shotters, who are *not* accountable for the work, and *don't* own the ideas they are shipping. Don't just do the right thing, make sure that you're nowhere near the line—that way it's easier for us to catch the cheats.
Perhaps you think we should come up with some clever means of parsing out just who is responsible for what, so you can get credit for what you did while still using AI for your personal gain. But why should we as a community bear all these costs in parsing good from bad behaviour, when there's a cheaply available alternative which is that you just don't use AI to write? So you're not a great writer? It's a skill that can be learned. So it's not your language? Well, sorry to say but plenty of people publish in a second language. No need to bring the system down to accommodate that. Nobody is expecting you to write like George Orwell or Cormac McCarthy. Dry scientific prose is fine if you can't muster something more engaging.
But what about all the ways AI could advance research, help push back the frontiers of knowledge? Nothing is stopping you from using AI in your research process! There are so many ways that you can experiment, so many resources you can draw on, while still being responsible for writing the work that you want others to read. That costly signal is crucial. It says that you have at least some mastery of the work you've submitted, that you have some ownership over it, and that you can be accountable for every word. Of course it's possible to write up a paper where each of these three is attenuated, but that's the kind of false negative we'll just have to live with (and frankly I doubt that many people will want to do "meat robot" except as a bit; it does not seem a fun way to spend one's time).
Still not persuaded? Ok fine. Make your own journal, your own track, your own community of people with their LLMs writing for each other. Don't pollute the commons for the sake of private gain. And tell people. We know anyway, but passing AI writing off as your own is not cool, it's like farting in an elevator and denying it was you.
I literally donated an organ for EA reasons and still wouldn't accept the label, but I think after all these years it's finally time to drop the "-adjacent". Still doesn't mean I'm a utilitarian, though.
pequoddity retweeted
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
