Tom Fehring

You can't just say you're going to steelman and then strawman! You set up this fully autonomous facility that could physically engineer all viruses and even manufacture its own DNA sequences. Then you essentially say "but we screen sequences" and "this lab cannot exist today." No one is arguing that you would need a facility like this. This is a bad argument as such a facility would indeed cost $100M, and we know that synthesizing a complex pox virus today in a university lab costs <$100k. You also say "there is no such thing as 1 lab that can synthesize all conceivable viruses." First, again, you do not need this. Second, different viruses do not typically change the fundamental lab equipment you would need –– just inputs like cell lines and reagents. (e.g. If you can make smallpox, you don't really need different equipment for ebola.) These are sometimes difficult to get, but motivated adversaries have typically been able to do so. (See: any state-level bioweapons program, other well-resourced groups.) Your two arguments against the steelman you set up are (1) that getting inputs is hard because we screen DNA sequences and (2) the science of creating an infectious virus is hard. Below are my counters. (1) Screening: Screening is currently not mandatory. Many providers do not screen, or screen insufficiently. We also do not need advanced AI to evade some of the screening we do have –– techniques like simply splitting orders between different providers have been shown to evade existing protocols. Even if screening is mandated, with controls on new benchtop synthesizers, it will still be possible to get around this. For more, see: ifp.org/how-to-secure-the-dn… (2) Virology is hard: Yes, virology is hard and there are physical barriers. There are several reasons why I do not think this means we should not be worried about bioweapons of mass destruction. First, sufficiently advanced AI is likely to change this. You do not need an automated lab for AI to provide benchwork uplift. I expect that eventually AI will also significantly improve our ability to use animal models, or even computational models, to predict in-human behavior. (And this is a good thing! Innovation will enable us to advance medicine far beyond what we can do today.) Second, timing is not a major concern for a motivated threat actor. If it takes a few years to design the right viruses, so be it. There is evidence that research consistent with gain-of-function work is going on as I write this, with at least some input from US frontier models, considering Anthropic's latest report. Finally, enabling biotechnology will improve, likely faster than normal because of AI. I will repeat this excerpt from a recent substack piece of mine: "This particular opinion is actually highly pessimistic about the rate of growth of technology. Many things used to be difficult that are no longer so. They become accessible as technology advances. Synthetic biology is not an exception to this, as we have watched costs fall and accessibility rise without fail since its inception. Before 2012, editing a specific site in a genome meant using zinc finger nucleases: custom-engineered proteins that cost $5,000+ to order, and were never widely adopted because they were so difficult to engineer well. CRISPR replaced this. You order the RNA guide sequence and buy the rest off the shelf, with total cost being as low as $30. This is a strong example of an engineering capability that was once siloed to a handful of specialist labs and is now an undergraduate exercise. Synthesis costs have also plummeted. Synthesizing the poliovirus genome cost about $300,000 in 2002, and the materials would cost under $1,000 today." Fearmongering is bad, but we don't need more AI uplift than we have now for bioweapons to be a major concern. There are active state weapons programs and well-resourced terror groups exploring bioweapons. These groups have already overcome many of the physical barriers, and you have granted that a model could propose a viral genome to synthesize, it could be synthesizable, and it could be infectious. I believe this is cause for alarm and urgent action.