AI-Designed threats and Biosecurity: A Growing Concern
A recent study has highlighted a potential vulnerability in current biosecurity measures: artificially intelligent (AI) systems can design perilous DNA sequences that may evade detection. The research demonstrates AI’s ability too identify and exploit weaknesses in systems designed to prevent the creation of biological weapons, raising concerns among scientists.
“My overall reaction was favorable,” says arturo Casadevall, a microbiologist and immunologist at Johns Hopkins University. “Here we have a system in which we are identifying vulnerabilities. And what you’re seeing is an attempt to correct the known vulnerabilities.” However, Casadevall cautions, “what vulnerabilities don’t we know about that will require future corrections?”
The study did not involve laboratory creation of the AI-designed proteins to verify if they would function as intended, mimicking the activity of biological threats. Casadevall emphasizes the importance of such “reality checks” as society confronts this emerging threat, acknowledging the potential complications posed by international treaties prohibiting biological weapons development.
This isn’t an isolated incident. Several years ago, another research team explored AI’s potential for misuse in generating novel molecules resembling nerve agents. Within six hours, the AI tool produced 40,000 molecules meeting the specified criteria. The AI not only identified known agents like VX but also designed numerous previously unknown molecules predicted to be even more toxic. The researchers refrained from publishing the chemical structures or synthesizing them in a lab, deeming them “way too dangerous,” according to David Relman, a researcher at Stanford University. “They simply said, we’re telling you all about this as a warning.”
Relman views the latest study, focused on evading security screening, as commendable but also indicative of a larger problem. “I think it leaves us dangling and wondering, ‘Well, what exactly are we supposed to do?'” he says. “How do we get ahead of a freight train that is just evermore accelerating and racing down the tracks, in danger of careening off the tracks?”
Despite these concerns, some biosecurity experts remain cautiously optimistic. James Diggans, head of policy and biosecurity at Twist Bioscience and chair of the International Gene Synthesis Consortium, notes that his company, a major provider of custom DNA, has referred fewer than five orders to law enforcement in the past ten years.
“this is an incredibly rare thing,” Diggans states. “In the cybersecurity world, you have a host of actors that are trying to access systems.That is not the case in biotech. The real number of people who are really trying to create misuse might potentially be very close to zero. And so I think these systems are an important bulwark against that, but we should all find comfort in the fact that this is not a common scenario.”
