Eric Neyman

Recently, a draft leaked of federal bill text being circulated by Anthropic. The bill does have some good elements, many of which have been seen before in state legislation (for example in Massachusetts) that Anthropic also supported in the past. I previously wrote about many of those elements here: xitter.gq/Thomas_Woodside/status…. Overall, however, this bill would be bad if passed into law, and I believe that circulating it has been counterproductive for federal AI safety discussions. I wish that Anthropic were embracing something closer to our policy suggestions here: secureaiproject.org/californ…. Here are my concerns about the bill’s content in more detail: 1. No minimum safety standards: The bill does not authorize any agency to create minimum safety standards requiring AI developers to implement any practice. This should be an important component of any federal framework; otherwise, all decisions are made purely by ad hoc court orders, which is not a good way to ensure safety. It does allow the agency to designate (but not create) security (not safety) standards. 2. Weak “blocking orders:” the government can only block a model if a risk is imminent, and the bar for showing that is high. There are many procedural requirements, such as the requirement that the agency head personally certify a lot of the filing, that would make it challenging to use this blocking power. In my view, an agency should be able to issue orders that take effect immediately if there is an imminent need for action (an imminent need for action differs from an imminent risk because it would include instances when an action is necessary soon to stop a risk, even if the harm itself would occur at an indeterminate point in the future). The agency should not have to wait for a court in such cases, although a court should of course be able to review quickly. 3. Evaluator shopping: We’ve previously said that at least some evaluators need to be selected by the government, not just chosen by the companies they are evaluating. This bill doesn’t even provide an accreditation system for evaluators, much less government selection. Instead, it provides only a rating system, which developers can feel free to ignore, except once the agency is doing a blocking order proceeding. 4. No “pacing:” This bill doesn’t go far enough in terms of pacing. If Anthropic does not embrace robust minimum standards, we think it’s especially important that something like compute-pacing be included. See our pacing proposal here: secureaiproject.org/californ…. 5. No embedded evaluators: There isn’t a requirement that evaluators be embedded at AI companies or that they continuously assess risk. They do have a requirement to produce rapid reports in certain circumstances, but without a duty to actually constantly assess whether they actually should produce a rapid report, it’s unclear whether they’ll do so. 6. No dispute resolution process: If an evaluator lacks information that it needs from a developer and the developer refuses to provide it, there isn’t a clear process to resolve the dispute. For example, there could be a process that permits the agency to order the information handed over, with a court review if a party disagrees. At present, the agency has to bring an action for a violation, which is very time consuming and would likely only happen in egregious cases. 7. Weak incident reporting: The definition of what is required to be reported is written very narrowly. It would not even cover incidents like Hugging Face unless they cause $250,000+ in damages (only the number of such incidents would have to be reported annually). 8. No preventing coercive control of government/secret loyalties: The bill doesn’t have provisions aimed at preventing insiders at AI companies from using powerful AI systems to coercively take control of government or otherwise provide themselves with significant advances. 10. No reporting of AI R&D automation: While covered as a risk factor, there are no reporting requirements to allow the public to understand how AI is being used to automate AI research and development unless the developer assesses this poses another “enumerated risk.” To be clear, I don’t think it’s necessary for every draft bill to have all of the above policies in it. That would be an unrealistic standard. It’s possible to have narrow, tight bills focused on particular problems that make a solid contribution to the discourse. That said, not everything in this bill is executed well, such as the blocking orders. More importantly, because it includes preemption, floating a bill this weak will likely have a detrimental effect on the political discourse. When a bill includes state preemption, it must be interpreted as the proposer’s view of what is “worth” that preemption. This is how everyone in DC will interpret it, so the proposer needs to propose it in that light. Anthropic is seen by many as being in favor of strong regulation (and state regulation), and portrays themselves that way. Their bill draft could give the false impression that this is what strong regulation looks like, anchoring future discussions and negotiations in a very suboptimal place. I do not think that the policy Anthropic has put forward is worth preempting all state frontier AI disclosure, evaluation, incident reporting, and possibly (the drafting seems to contain an error) model blocking laws. Anthropic representatives have said they circulated this in part because “it’s important to get early feedback from policymakers.” However, in my view, floating language like this signals to policymakers that Anthropic is prepared to support it as is. And as is, it is a bad bill.