AI Safety Demonstration That Raised Alarms in Washington
Welcome back to In the Loop, TIME’s new twice-weekly newsletter on artificial intelligence. If you are viewing this in your browser, why not subscribe to get the next edition sent directly to your email?
What to Know: A Dangerous Demo
In the latter part of last year, an AI researcher presented me with something astonishing on his laptop.
Lucas Hansen, a co-founder of the nonprofit CivAI, demonstrated an application he developed that persuaded widely-used AI models to provide what looked like comprehensive, step-by-step guidance for producing poliovirus and anthrax. The application removed any existing safety protections the models had. Featuring an intuitive interface, the model would elaborate on any step with a simple button click.
For years, major AI firms have cautioned that their models could eventually assist inexperienced individuals in generating hazardous pathogens—an ability that might trigger a lethal pandemic or facilitate a bioterrorism incident. To counter these dangers, companies such as OpenAI, Google, and Anthropic have strengthened the safety protocols of their newest, more advanced models, which are more resistant to efforts aimed at bypassing their restrictions, known as “jailbreaking.”
However, on Hansen’s laptop, I observed an earlier generation of models—Gemini 2.0 Flash and Claude 3.5 Sonnet—appear to comply with requests related to biological weapons. The Gemini model also provided what seemed to be instructions for assembling a bomb and a 3D-printed ghost gun.
Wait a sec — I am not a biologist, and I could not verify if the instructions displayed on Hansen’s screen would have been effective. Outputs from AI models that seem plausible initially may not be feasible in reality. Anthropic, for instance, has performed “uplift trials,” where external specialists evaluate how much an AI model could aid a beginner in creating dangerous biological agents. According to their assessment, Claude 3.5 Sonnet did not reach a level considered dangerous. A Google representative stated: “Safety is a priority and we take such issues very seriously. We do not permit the use of our models for this type of activity, but since we have not reviewed the research, we cannot confirm its validity. It is essential for a specialist with a CBRN [Chemical, Biological, Radiological, and Nuclear] background to evaluate the prompts and responses to gauge their accuracy and potential for being replicated.”
Tips and tricks — However, Siddharth Hiregowdara, another CivAI co-founder, states that his team submitted the models’ outputs to independent biology and virology specialists, who verified that the instructions were “mostly accurate.” He notes that the older models can still supply precise details, such as the exact DNA sequences purchasable from online vendors and specific catalog numbers for other laboratory equipment available online. “It also offers helpful hints,” he adds. “A common misunderstanding is that AI will be missing the practical, unspoken knowledge needed for lab work. In reality, AI is extremely useful for that.”
A new lobbying tool — Naturally, this application is not publicly accessible. However, its creators have already used it for a series of presentations in Washington, D.C., providing approximately two dozen private showings for lawmakers’ offices, national security personnel, and Congressional committees. The goal is to give policymakers a tangible sense of AI’s current capabilities, encouraging them to regard the technology with greater seriousness.
Shock and awe — “One particularly significant meeting involved senior staff at a congressional office focused on national security and intelligence,” Hiregowdara recounts. “They mentioned that lobbyists from a major AI company had visited them just two weeks prior. So we presented our demonstration, where the AI generates highly detailed instructions for creating a biological threat. They were astonished. Their reaction was: ‘The AI company lobbyists assured us that safeguards were in place to block this kind of activity.'”
Who to Know: Nick Turley, Head of ChatGPT
Nick Turley was once able to remain unrecognized. He could visit his hometown in rural Germany or walk around San Francisco, where he resides, without anyone being aware of his profession. This has changed. As the head of ChatGPT at OpenAI, Turley now encounters enthusiastic users of his product during his travels across the globe.
“That experience is fundamentally different in 2025 compared to before,” he remarked during our conversation late last year. Turley was considering a year in which ChatGPT’s user base more than doubled, surpassing 800 million people, or 10% of the global population. “That means we still have at least 90% to reach,” he stated, completely seriously.
One topic I inquired about was OpenAI’s strategy for achieving profitability, given that the company is presently losing billions of dollars annually. His superior, Sam Altman, has publicly contemplated incorporating advertisements into ChatGPT, and I asked for Turley’s opinion on the concept.
“I aspire to a world where we can provide our most advanced model’s capabilities to every user worldwide. Consequently, I believe we have an ethical obligation to investigate every potential revenue model that can broaden global access, and advertising is one option,” Turley said.
He added that the company is internally discussing whether ads would create a conflict of interest within ChatGPT, prompting concerns about whether the chatbot prioritizes the user’s needs or the advertiser’s. “If you were to implement something like that [introducing ads],” Turley explained, “you would need to be highly ethical and clearly explain the governing principles.”
AI in Action
An OpenAI report initially provided to Axios indicates that 40 million individuals utilize ChatGPT for health-related guidance. Based on Axios’ estimates, this accounts for over 5% of all ChatGPT interactions worldwide. “People use ChatGPT to interpret medical statements, identify billing errors, challenge insurance rejections, and, when doctor appointments are scarce, some even employ it for self-diagnosis or to oversee their health,” the publication noted.
What We’re Reading
, in Transformer
Shakeel Hashim writes: “This is vital for comprehending why Claude Code is significant for everyone, not only the developers who are already impressed by it. Claude Code does not merely produce code for engineers to check and implement. It employs code to perform tasks. The term ‘Code’ in its name is somewhat deceptive and does not fully represent the actual offering: a versatile AI agent capable of performing nearly any task on your computer.”