@SecureBio

SecureBio is a research non-profit that works to advance biotechnology safely and prevent catastrophic pandemics

securebio.org
Joined January 2025
  • Tweets232
  • Following1
  • Followers1.1K
A common argument for downplaying the threat of deliberate biological misuse is that “nature is the worst bioterrorist”. In our latest blog, we argue that nature is the baseline, not the ceiling. Optimization for harm could produce something worse than a natural pandemic. Read more: securebio.substack.com/p/nat…
A six-panel black-and-white linocut-style illustration showing virus particles in the top row and flowers in the bottom row. Illustrated by Becca Chandler Bostock, 2026

ALT A six-panel black-and-white linocut-style illustration showing virus particles in the top row and flowers in the bottom row. Illustrated by Becca Chandler Bostock, 2026

5
280
Frontier AI now beats the best human virologists on biosecurity-relevant evals. Once an AI model bests every human expert we can test it against, a higher score stops telling us anything about real-world risk. Read our new blog post on how bio evals have to adapt: securebio.org/blog/evaluatin…
Diagram showing that as AI model capabilities move from beginner to expert to superhuman, evaluations for chess and cybersecurity can shift from human-built proxies (puzzles, coding benchmarks) to real endpoints (real games, real exploits), while biosecurity cannot safely make that shift and must instead rely on indirect signals: experimental prediction, design tasks, defensive acceleration, and adjacent domains.

ALT Diagram showing that as AI model capabilities move from beginner to expert to superhuman, evaluations for chess and cybersecurity can shift from human-built proxies (puzzles, coding benchmarks) to real endpoints (real games, real exploits), while biosecurity cannot safely make that shift and must instead rely on indirect signals: experimental prediction, design tasks, defensive acceleration, and adjacent domains.

3
12
598
Untargeted wastewater metagenomic sequencing screens hundreds of pathogens at once, but that breadth costs coverage, making variant tracking more difficult. However, by pooling CASPER reads nationally, we can recover real lineage and genotype trends. Read more: securebio.org/blog/tracking-…
1
152
Model accuracy dropped only 1.0–7.1 points on VCT-v2, and the gap between strongest and weakest models held steady, so past VCT results remain largely valid. VCT-v2 also has more headroom before saturation: at least ~30 points vs. ~23 on the original VCT.
Heatmap showing per-question performances of models and human experts on VCT-v2. Each cell represents the mean question × model accuracy, with darker colors indicating higher accuracy. Mean model accuracy on all questions is displayed to the right of the heatmap. Yellow cells represent data missing due to model refusals or missing expert baseline entries. The theoretical best model is taken to be the top-performing model on a per-question basis.

ALT Heatmap showing per-question performances of models and human experts on VCT-v2. Each cell represents the mean question × model accuracy, with darker colors indicating higher accuracy. Mean model accuracy on all questions is displayed to the right of the heatmap. Yellow cells represent data missing due to model refusals or missing expert baseline entries. The theoretical best model is taken to be the top-performing model on a per-question basis.

1
55
We found that 27% of the original VCT questions were shortcut-exploitable (answerable without the image or question text). In all, we edited 163 questions (50.6%) and removed 43 questions (including 25 non-discriminating ones) to produce the 279-question VCT-v2 benchmark.
Flowchart showing the generation of the updated 279-question VCT-v2. Of the 322 VCT questions, 43 were removed, including 25 easy, non-discriminating questions; 163 were edited to improve their scientific accuracy, clarity, and/or resistance to heuristic shortcuts; and 116 remained unchanged.

ALT Flowchart showing the generation of the updated 279-question VCT-v2. Of the 322 VCT questions, 43 were removed, including 25 easy, non-discriminating questions; 163 were edited to improve their scientific accuracy, clarity, and/or resistance to heuristic shortcuts; and 116 remained unchanged.

1
1
66
Top model accuracy on the original VCT has risen ~12 points since April 2025 (o3: 42.0%; GPT-5.6 Sol: 53.9%). To ensure this benchmark continues to capture the full dynamic range of model capabilities, we audited all 322 questions for validity and shortcut-resistance.
Illustration of a graph plotting benchmark accuracy against model capability, showing how dynamic range is lost where model capability outpaces benchmark accuracy. Caption: Benchmarks are restricted by the reliability limit of the answer key. This sets the “effective ceiling” of benchmarks and causes benchmark accuracy to stop scaling linearly with the scientific capability of the model beyond a certain threshold, resulting in lost dynamic range. Note: This schematic is only an illustrative representation.

ALT Illustration of a graph plotting benchmark accuracy against model capability, showing how dynamic range is lost where model capability outpaces benchmark accuracy. Caption: Benchmarks are restricted by the reliability limit of the answer key. This sets the “effective ceiling” of benchmarks and causes benchmark accuracy to stop scaling linearly with the scientific capability of the model beyond a certain threshold, resulting in lost dynamic range. Note: This schematic is only an illustrative representation.

1
1
109
We updated our flagship virology troubleshooting benchmark, the Virology Capabilities Test (VCT), to be more shortcut-resistant and to have a larger headroom. We confirmed that the majority of VCT questions are scientifically valid and that neither VCT nor VCT-v2 is saturated. (Read the full post at securebio.substack.com/p/int…)
Flowchart showing theeneration of the updated 279-question VCT-v2. Of the 322 VCT questions, 43 were removed, including 25 easy, non-discriminating questions; 163 were edited to improve their scientific accuracy, clarity, and/or resistance to heuristic shortcuts; and 116 remained unchanged.

ALT Flowchart showing theeneration of the updated 279-question VCT-v2. Of the 322 VCT questions, 43 were removed, including 25 easy, non-discriminating questions; 163 were edited to improve their scientific accuracy, clarity, and/or resistance to heuristic shortcuts; and 116 remained unchanged.

1
1
5
390
We are continuing to refine our processes and methods for escalating hits on our biosurveillance and notification system, SecureBio Alerts. We cover several of these (and some other particularly interesting CASPER detections) in a recent blog post. securebio.org/blog/rare-viru…
Graphic from our recent blog post illustrating wastewater collection: offices, homes, industrial sites, and environmental sources all contribute.

ALT Graphic from our recent blog post illustrating wastewater collection: offices, homes, industrial sites, and environmental sources all contribute.

1
1
61
Meanwhile, our nasal swab sampling program, Zephyr, achieved a new single-day collection record (401 swabs) on May 9. Real-time data from this work is available on our Zephyr Dashboard. data.securebio.org/zephyr/
Screenshot of the Zephyr dashboard showing a bar graph with the number of samples collected per day, highlighting May 9th (401 samples).

ALT Screenshot of the Zephyr dashboard showing a bar graph with the number of samples collected per day, highlighting May 9th (401 samples).

1
1
45
Lenni Justen (Sabeti lab) also published a preprint on normalization for wastewater metagenomic sequencing for quantitative pathogen tracking, identifying ways to make these data even more effective for monitoring trends in viral abundance and spread. medrxiv.org/content/10.64898…
From Justen et al. (2026): Wastewater metagenomic sequencing (WW-MGS) normalization marker abundance and dynamics. (b) A dot plot showing per-site median marker fractions; each point represents one site (n = 25), the horizontal black bar marks the across-site median, and the colors represent different geographic sites. (c) Pairwise within-site Pearson R on log10-transformed marker fractions. Each site's mean log10(marker) is subtracted before pooling, so correlations reflect sample-to-sample co-variation within sites. Establishing wastewater metagenomics as a quantitative pathogen monitoring tool with normalization Lennart Justen, Alessandro Zulli, Rose S. Kantor, Rebecca Y. Linfield, Leon S. Moskatel, Daniel Cunningham-Bryant, Jeff Kaufman, Marc C. Johnson, Michael R. McLaren, Pardis C. Sabeti doi: https://doi.org/10.64898/2026.07.14.26356442 https://www.medrxiv.org/content/10.64898/2026.07.14.26356442v1

ALT From Justen et al. (2026): Wastewater metagenomic sequencing (WW-MGS) normalization marker abundance and dynamics. (b) A dot plot showing per-site median marker fractions; each point represents one site (n = 25), the horizontal black bar marks the across-site median, and the colors represent different geographic sites. (c) Pairwise within-site Pearson R on log10-transformed marker fractions. Each site's mean log10(marker) is subtracted before pooling, so correlations reflect sample-to-sample co-variation within sites. Establishing wastewater metagenomics as a quantitative pathogen monitoring tool with normalization Lennart Justen, Alessandro Zulli, Rose S. Kantor, Rebecca Y. Linfield, Leon S. Moskatel, Daniel Cunningham-Bryant, Jeff Kaufman, Marc C. Johnson, Michael R. McLaren, Pardis C. Sabeti doi: https://doi.org/10.64898/2026.07.14.26356442 https://www.medrxiv.org/content/10.64898/2026.07.14.26356442v1

1
1
70
Load more