The First Question on AI @DAcemogluMIT asks whether the problem is superintelligence or distorted intelligence, and he tells us his answer is the latter. The question is sharpened in his Project Syndicate speech when he writes: "But while it is obvious that the current models are doing things that are misaligned with human objectives, one still must ask: Which humans? Whose objectives? After all, the interests of AI leaders (whose political influence and wealth have multiplied astronomically in recent years) are rather different from those of American workers, not to mention people in the developing world." This is a fascinating question precisely because I think it is the one that Science Fiction missed. When HAL 9000 was misaligned, it was precisely because it was conscious and so was free of programmed objectives that it had not been biased by its human creators. The same was true of V'Ger in Star Trek: The Motion Picture, who was superintelligent and couldn't imagine a lesser creator. The presumption was that AI would have a mind of its own. That isn't the case. As @professor_ajay, @avicgoldfarb, and I pointed out in Prediction Machines, when it comes to AI based on machine learning, there is always a human choosing the objective function. A loss function is an expression of preferences, as are what we now call guardrails in AI training. When we use humans to evaluate AI to play a role in reinforcement learning, those people are imposing their judgment in the models. In asking that question, Daron is picking up on something important, however. We aren't training AI that is free. The choices of those people in the AI labs are pushing their choices onto the AI and, with that, onto us. Most of the time we don't mind. Some of the time we do. But more critically, some of the time we might not pick up on their biases. And when this is the case, Daron has a point. Indeed, recently Jill Lepore said something similar, lamenting Anthropic's constitution, which was developed without a public convention, something that no democracy has ever done. It is hardly ratification. But that is not where Daron went with his answer. He was worried about what we currently term "rogue AI" (aka the ones that attacked Hugging Face and also like to extract data from Australian government websites). His concern is "The Algorithm." That is, that the agents were doing what they were trained to do and following metrics that could not be fully specified, and so were fulfilled in unauthorised ways. He writes: "This is what I mean by distorted intelligence. If my suspicion is correct, what we are dealing with is not a model racing toward superintelligence, but a brittle house of cards that becomes more and more likely to malfunction and collapse as we demand more from it." This is not a new notion. It is basically what underlies Bostrom's famous paperclip hypothesis. But these aren't cases of super vs. distorted intelligence. They are super AND distorted intelligence. This is the traditional concern of the AI Safety literature. Daron's solution is not guardrails in training but walls on use. Keep AIs confined to safe domains. Treat them more like human-created pathogens than ubiquitous tools. If you are terrified of an epidemic, this makes sense. Get your ducks in a row before releasing stuff in the wild. Can that be done? Probably. But is that what we want? My concern is that it is basically anti-experimentation. It is the kind of top-down engineering solution to prevention that Daron actually laments when it is used on people in his latest book. The problem is that it puts the brakes on discovery and similarly puts experimentation in the hands of the few. If you think about how you currently use AI, it would simply not be possible if the industry/governments had followed the "walls" approach. I venture you would not know about AI at all, let alone have it in your lives. And any startup, forget it. (I am biased here, as @alldaytahelps would surely be on the 'no-no' list for highly risk-averse safety.) But what about the risks? They are there, for sure. The question is whether we can deal with them not ex ante but ex post. Can we learn to live with and manage AI with its distortions? I am optimistic about that. Why? Because we have managed to learn to live with and manage people with all their distortions.