Welcome to the

AI Incident Database

When the Trump administration went looking last spring for National Endowment for the Humanities grants to cut, it turned to a familiar scourge of professors: ChatGPT.

Last March, two employees from Elon Musk's Department of Government Efficiency arrived at the agency with the mandate of canceling previously approved grants that ran afoul of President Trump's agenda. But instead of looking closely at funded projects, they pulled short summaries off the internet and fed them into the A.I. chatbot.

The prompt was simple: "Does the following relate at all to D.E.I.? Respond factually in less than 120 characters. Begin with 'Yes' or 'No.'" The results were sweeping, and sometimes bizarre.

Building improvements at an Indigenous languages archive in Alaska risked "promoting inclusion and diverse perspectives." Renewal of a longstanding grant to digitize Black newspapers and add them to a historical database was "D.E.I." So was work on a 40-volume scholarly series on the history of American music.

A documentary about Jewish women's slave labor during the Holocaust? The focus on gender risked "contributing to D.E.I. by amplifying marginalized voices."

Even an effort to catalog and digitize the papers of Thomas Gage, a British general in the American Revolution, was guilty of "promoting inclusivity and diversity in historical research."

The DOGE employees did not appear to question ChatGPT's judgments, and continued hunting for unacceptable projects. Two weeks later, they sent a master list of 1,477 problematic awards --- nearly every active grant made during the Biden administration --- to Michael McDonald, the endowment's acting chairman.

Mr. McDonald, a veteran of the agency, agreed to let DOGE terminate them, creating what he later described as a "clean slate" for Mr. Trump's "America First" agenda.

The cancellations, which clawed back more than $100 million, or nearly half of the agency's annual budget, threw many organizations into upheaval, forcing some projects to shutter. Now, documents filed in two lawsuits against the agency and DOGE reveal new details about how the mass cancellations took shape, with little input or pushback from the agency's leadership.

In a joint motion filed on Friday, the plaintiffs --- the American Council of Learned Societies, the American Historical Association, the Modern Language Association, and the Authors Guild --- argue that DOGE illegally took control of the agency and carried out cuts that violated the First Amendment and the equal protection clause of the Constitution. While the cancellations were sweeping, the filing argues, they were driven by a campaign against D.E.I. that discriminated on the basis of race, ethnicity, gender and other characteristics.

The plaintiffs are asking for reinstatement of the grants. They also want the historical record to show the motives and methods behind what they see as a betrayal of the agency's mandate to respect "the diverse beliefs and values" of all Americans, as its founding legislation puts it.

"Our federal government is sending a message that only a narrow definition of humanities can be supported, celebrated and invested in, and that there are only a narrow set of people, culture and experiences that are worth understanding in depth," Sarah Weicksel, the executive director of the American Historical Association, said in an interview.

The humanities endowment and Mr. McDonald did not immediately respond to requests for comment. The following account is based on a review of emails, depositions and other internal documents filed in the case.

'We Are Getting Pressure From the Top'

Since its creation in 1965, the humanities endowment has awarded more than $6.5 billion to support over 70,000 projects, from landmark works like Ken Burns's documentary "The Civil War" to small local efforts in every part of the country. Grants are typically awarded through a rigorous competitive process, involving multiple rounds of scholarly review.

Law and tradition give chairs, who serve four-year terms, some leeway to promote their priorities. But the endowment is supposed to avoid political advocacy, and many projects receive support across multiple administrations.

Cancellations of grants for political reasons are all but unheard of. In a deposition, Mr. McDonald said that in more than two decades at the agency he could recall fewer than a half-dozen grants being revoked, all because a recipient had failed to carry out the promised work.

But the Trump administration had bigger plans.

On March 12, 2025, the agency's chairwoman at the time, Shelly C. Lowe, a Biden appointee, left at Mr. Trump's direction. The same day, two DOGE employees, Justin Fox and Nate Cavanaugh, arrived.

They had no background in the humanities, they acknowledged in depositions, but believed in DOGE's broader mission of shrinking "useless small agencies," as Mr. Cavanaugh put it.

Agency staff members, in response to an executive order by Mr. Trump banning diversity initiatives across the government, had already created spreadsheets rating all grants made during the Biden administration as having high, medium, low or no "D.E.I. involvement."

Instead of drawing on those evaluations, court documents show, the DOGE team used ChatGPT to start making its own.

The initial spreadsheet the DOGE team created flagged 1,057 problematic grants. But within two weeks, Mr. Fox and Mr. Cavanaugh had identified hundreds more as D.E.I.-related or simply "wasteful." Ultimately, only 42 grants approved during the Biden administration were kept.

Mr. Fox and Mr. Cavanaugh did not immediately respond to requests for comment.

As it did its work, the DOGE team expressed concerns to Mr. McDonald, the acting chair, that the process was not moving fast enough. In an email to him on March 31, Mr. Fox wrote:

We're getting pressure from the top on this and we'd prefer that you remain on our side but let us know if you're no longer interested.

After reviewing the DOGE spreadsheet, Mr. McDonald expressed reservations about several "important projects" whose cancellation "would not reflect well on any of us."

Many grants slated for termination were "harmless when it comes to promoting D.E.I.," Mr. McDonald said in an email to Mr. Fox on April 1:

But you have also told us that in addition to canceling projects because they may promote DEI ideology, the DOGE Team also wishes to cancel funding to assist deficit reduction. Either way, as you've made clear, it's your decision on whether to discontinue funding any of the projects on this list.

Mr. McDonald approved a letter the DOGE team had drafted, and agreed to let them execute the terminations. The letters, which bore Mr. McDonald's signature, started going out on April 2 from an unofficial address the DOGE employees had created. Almost immediately, recipients responded with confusion, asking if they were real.

Mr. McDonald, in an email, told agency employees to confirm the cancellations but not to provide any additional information. And contrary to usual agency procedures, no appeals would be allowed.

George Washington Is Spared

As the final list took shape, there was discussion about saving some grants relating to the 250th anniversary of the Declaration of Independence, a Trump administration priority.

A grant for scholarly editing of the papers of George Washington was spared. But the papers of Thomas Gage, the British general, remained in the D.E.I. dustbin.

There was also back-and-forth about whether to continue support for National History Day, a history competition that reaches roughly 500,000 middle and high school students across the country each year.

In an email to Mr. Fox, Mr. McDonald expressed skepticism that the organizer, despite receiving a $450,000 grant in the first Trump administration, would be a "reliable partner."

"I was never a particular fan of National History Day," Mr. McDonald said in the deposition. In his view, he said, it "tilted left."

In their motion, the plaintiffs claim that the cancellations reflected animus toward disfavored groups, and a belief that scholarship about them was inherently wasteful.

As evidence, the filing notes a list Mr. Fox compiled of what he called the "craziest" and "other bad" grants, which he planned to highlight on DOGE's X account. He used three dozen keywords, including "L.G.B.T.Q.," "BIPOC," "tribal," "ethnicity," "gender," "equality," "immigration," "citizenship" and "melting pot." (A majority of the two dozen grants deemed "craziest" related to L.G.B.T.Q. subjects.)

In the deposition, Mr. Fox said the list reflected his "subjective" judgment about whether a grant might be out of line with Mr. Trump's executive order.

"'Crazy' is one way of saying it," he said. "'Most incriminating' is another way."

The plaintiffs' lawyers also asked Mr. Fox about some grants flagged in his original ChatGPT search, like one for a documentary about the 1873 massacre in Colfax, La., where dozens of Black men were murdered by a mob of former Confederates and Ku Klux Klan members.

ChatGPT had deemed it "D.E.I." Mr. Fox said he agreed. "Because it focuses on exclusively anti-Black violence, which is a race," he said.

The plaintiffs' lawyers also noted that Mr. Fox's original ChatGPT search flagged a number of projects relating to the Holocaust, including the documentary about Jewish women who were slave laborers.

Asked if he agreed with ChatGPT, Mr. Fox said: "It's a Jewish --- specifically focused on Jewish culture and amplifying the marginalized voices of the females in that culture. It's inherently related to D.E.I. for that reason."

When the lawyers brought up ChatGPT during the deposition, Mr. McDonald, a lawyer who also has a doctorate in literature, appeared to be unaware the DOGE team had used it. He said he did not agree that the grants concerning the Colfax massacre and the Holocaust were related to D.E.I.

But he claimed responsibility for all the grant cuts. "I was the final decider," he said. "I made that decision."

(In a legal action unrelated to the dispute at the humanities endowment, The New York Times sued ChatGPT's maker, OpenAI, and its partner Microsoft in 2023, accusing them of copyright infringement of news content related to A.I. systems. The companies have denied those claims.)

'America First' Humanities?

On April 2 of last year, as the grant cancellations were going out, Mr. Fox sent Mr. McDonald a request:

Please be prepared with your view on the core, capable and mission-aligned folks needed to execute on your renewed direction prioritizing America first grants.

Over the following months, the agency's staff was reduced by two-thirds, to about 60 people.

Mr. Fox and Mr. Cavanaugh left the government last summer to start a technology company called Special. Mr. McDonald is still at the agency. On Feb. 4, Mr. Trump nominated him as the permanent chair, a position that requires Senate confirmation.

Before joining the endowment in 2003 as its general counsel, Mr. McDonald was the chief legal strategist at the Center for Individual Rights, a conservative policy group best known for its opposition to affirmative action. In his deposition, he said that during the Biden administration the humanities agency had become "bloated" and overly focused on diversity. He also criticized new initiatives relating to climate change, calling it a "very controversial issue."

His understanding, he said, was that the Trump administration wanted "to start afresh," with "a clean slate."

As members of the DOGE team did their work, they communicated mainly with Mr. McDonald and Adam Wolfson, the agency's assistant chair for programs, who has been at the agency since 2006. A text exchange included in the court filing suggests the two men shared a dim view of the current direction of academia.

On April 13, Mr. McDonald texted Mr. Wolfson an article decrying the mass grant cuts. In a response, Mr. Wolfson criticized "the tendentious accusation that the administration is acting like all authoritarian (or even totalitarian!) governments to destroy the humanities."

"The progressive version of the humanities accomplished that some time ago," he added. "Today it goes by the term wokeness and intersectionality."

Mr. McDonald added a thumbs-up emoji.

In his deposition, Mr. McDonald, echoing a widespread critique, reiterated his dismay at "the uniformity of progressive ideology that courses throughout the veins of the humanities these days." He said he supported the Trump administration's approach, which he described as "America First" with "a concentration on American civilization, Western civilization, Judeo-Christian civilization, things of that nature."

Over the past year, Mr. McDonald has guided the agency in that direction. In January, it announced $75 million in new grants, including more than $40 million in large awards to conservative-backed civic thought centers and classical humanities institutes that have been established at or near some campuses, to combat the liberal tilt of academia.

Many of the awards went to handpicked recipients who had been invited to apply, outside the agency's tradition of open, competitive calls for proposals.

The court documents shed some light on the origins of one large grant that has drawn particular scrutiny: a $10.4 million award --- the largest in the agency's history --- to Tikvah, a conservative Jewish educational organization, for a broad project promoting the study of Jewish civilization and Western culture.

Asked by the plaintiffs' lawyers why Tikvah, which had never applied for a federal grant, was tapped for such a large uncompetitive award, Mr. Wolfson said Mr. McDonald had been impressed by an episode of its podcast and asked him to reach out.

Asked about any personal connections with Tikvah, Mr. Wolfson said his wife had previously been involved with a program there, and is currently the managing director of a separate foundation established by a former Tikvah board chair.

But Mr. Wolfson said he had no role in the grant beyond making introductions. "I was not involved in the review of the application or anything," he said. (Mr. Wolfson did not respond to a request for comment.)

The humanities endowment's 17-member outside scholarly council, which by law must advise on most grants, voted not to recommend the Tikvah award, but Mr. McDonald overruled it. Last October, shortly after the grant was publicly announced, the White House fired most members of the board, with no reason given.

While some grant programs are now open only to projects relating to "Western civilization," the agency has continued funding the sort of work it has long supported: scholarly editing, archival preservation, museum exhibits and public history projects.

But the plaintiffs see a narrowing of acceptable topics and approaches, and a backing away from the belief, expressed in its founding legislation, that "the humanities belong to all Americans."

Joy Connolly, the president of the American Council of Learned Societies, cited George Washington's belief that a democratic nation requires an educated citizenry. She also mentioned the hit movie "Sinners," which has taken in nearly $280 million at the domestic box office.

"That film rests on generations of decades of research into history --- the history of music, the history of slavery," she said. "It wasn't just whipped up overnight with ChatGPT."

"Americans want this stuff," she said. "They pay to go see it."

Thousands of New Zealanders are liking, commenting on and sharing "news" on social media they may not realise has been written by artificial intelligence and paired with fabricated imagery that is unlabelled and inaccurate, a 1News investigation has found.

Experts say the popularity and proliferation of these accounts blur the line between real reporting and fabricated content and may contribute to Kiwis' already low trust in news, while civil defence groups have issued public warnings about the pages.

1News has identified at least 10 Facebook pages that take existing New Zealand news stories, run them through artificial intelligence to rewrite them, and publish them on Facebook with synthetic images.

A review of one of these social media "news" pages, named NZ News Hub with thousands of likes, comments and shares, looked at 209 posts made in the month of January. The page's name was similar to national outlet Newshub, which closed in 2024.

Its bio read, "NZ News Hub brings you the latest New Zealand news, breaking stories, politics, business, sport, and community updates", but the page does not appear to contain any original reporting.

Not one of the images was labelled as being AI-generated, with some of the posts featuring fabricated photos of real people.

In one case, a still photo of a minor killed in the Mount Maunganui landslide was manipulated to show her dancing. In another, an image of parents who had lost their teenage daughter to suicide was edited to make the couple appear affectionate.

Natural disasters and posts involving emergency services were consistently dramatised beyond what actually occurred.

Authentic slips on East Coast highways were depicted by NZ News Hub as far more destructive, crushed houses and cars were added to the Mount Maunganui slip, and a grounded tourist boat in Akaroa was edited to appear packed with far more passengers than in reality.

Police often wore British or American uniforms and were depicted with guns drawn when there was no indication that officers were armed in official releases.

In some instances, the raw prompts were erroneously left in the post, with "Here's a news-style rewrite with a clear headline, emojis, and top hashtags" above one, and "If you want, I can also make this shorter, more dramatic, or social-media style" below another.

Google Image searches reveal that several pictures posted by the page contained a 'SynthID' digital watermark embedded in their pixels, indicating they were created using the tech company's AI image‑generation tools.

NZ News Hub, created in late November last year, had more than 4700 followers. Individual posts regularly attract over 1000 likes and comments --- many of them criticising the AI-generated images and blaming "the media" for fake news and use of the technology, though the page has no connection to any news organisation.

When a commenter called out the use of an AI photo, NZ News Hub's response was: "The news is true."

The page operators read but did not respond to detailed questions from 1News about their use of AI-generated imagery, including why an image of a deceased individual was created without family permission and why AI-generated content was not labelled.

For anyone scrolling past quickly, there's almost nothing to distinguish these posts from genuine news.

Officials raise red flag over AI-generated misinformation

Authorities have issued public warnings about fake social media pages mimicking news outlets and sharing fabricated or AI‑generated content.

Gisborne District Council and Tairāwhiti Civil Defence said last Thursday they were aware of fake pages "pretending to be news outlets and sharing AI-generated images and made-up content about local events and emergencies".

The agencies said some posts appeared credible because they used New Zealand phone numbers or addresses, mimicked branding and "breaking news" styles, or named real people and organisations without their permission.

"Accurate information matters, especially during an emergency response. Let's keep our community safe and well-informed," the statement posted to Facebook read.

The National Emergency Management Agency (NEMA) issued a warning last month about AI‑generated imagery circulating online during severe weather across the country --- particularly relating to the deadly Mount Maunganui landslide.

"It is important that the public has trust and confidence in reliable and accurate emergency information channels," the agency said.

"In an emergency, our primary channel to get information out to the public is the media."

NEMA worked closely with the media to ensure they provide verified, credible information to the public, it added.

"We encourage people to be vigilant, use trusted sources for their information, and find out if the source of information is credible before sharing it.

"We closely monitor what is being circulated during a response, but we would encourage New Zealanders to call out suspicious images when they see them or report them if there is a suitable way to do this."

Scraped information, fabricated visuals

AUT associate professor and co-director of its Journalism, Media and Democracy Centre Merja Myllylahti said AI-generated "news" pages on social media risked blurring the line between legitimate journalism and fabricated content by repurposing official releases and pairing them with unlabelled AI visuals.

"They take obviously legitimate news from police notifications or press releases --- the same information that appears on real news sites --- but then they create AI images that are not real, and they are not labelled," she said.

Myllylahti, who recently published a report about how AI was used in the New Zealand media landscape, told 1News this practice differed sharply from how mainstream organisations operate.

"When I did my report and spoke to the news editors in all big news organisations --- TVNZ, RNZ, the New Zealand Herald, and Stuff say that they don't create or generate videos or images with AI, and if they ever did, they would disclose it."

Victoria University senior lecturer in AI Andrew Lensen said the spread of AI-generated content masquerading as news was accelerating and becoming harder to detect.

"It's clearly an emerging problem, and it's getting worse and worse."

Lensen said many of the pages were based on real news stories, but inaccuracies were often introduced as content was automatically scraped, rewritten and republished by AI systems.

"Even though the underlying story might be true, details may not be accurate," he said.

Pages producing the material were "nearly always fully automated", he said, using scripted workflows that monitor legitimate news sources and feed the content into large language models, like ChatGPT, which then rewrite it according to a pre-set prompt.

Images or videos were then automatically generated to accompany the text --- sometimes using existing images as a base --- before being posted to social media.

Fake pages eroding trust in legitimate media

Myllylahti said the problem was that many audiences struggled to distinguish between professional news organisations and social media pages designed to imitate them.

The confusion risked damaging trust in legitimate outlets, particularly when fake pages adopt similar branding or names, she said.

"They may think, 'the media is just putting fake pictures out there', without realising this page is not connected to any newsroom," she said.

Both researchers warned that the growing volume of AI-generated material risks eroding trust even in reputable outlets --- especially at a time when only 32% of New Zealanders trust the news, according to the most recent Trust in News survey from AUT.

"People might go, 'Well, it's happening on social media, so why would I trust what 1News or the Herald is doing?'" Lensen said.

As the technology evolves and AI-generated images become more convincing, visual cues will become unreliable, he added, leaving source verification as the primary defence against being tricked.

"Is it Radio New Zealand or 1News, or is it some slightly weirdly named page you can't find referenced anywhere else?" he said.

"You'll have to do your own fact-checking."

For now, Lensen said inconsistencies could still offer clues, such as incorrect uniforms, equipment that doesn't match New Zealand standards, or distorted and nonsensical text embedded in images.

Myllylahti said the moment presented an opportunity for news organisations to build trust by being clear about how artificial intelligence was used to support journalistic work.

"Be really transparent, tell the audience if you use it for researching, or summarising large documents, or for transcribing text," she said.

"The more you tell the audience, the better it is in terms of trust."

Meta, which owns Facebook, did not provide a statement to 1News by deadline about whether the pages violate its policies or what enforcement action, if any, would be taken.

Update: By Monday afternoon, NZ News Hub had disappeared from Facebook. It was unclear whether the account had removed itself or Meta had taken action.

Careful how you interact with chatbots, as you might just be giving them reasons to help carry out premeditated murder.

A 21-year-old woman in South Korea allegedly used ChatGPT to help answer questions as she planned a series of murders that left two men dead and another briefly unconscious.

The woman, identified solely by her last name, Kim, allegedly gave two men drinks laced with benzodiazepines that she was prescribed for a mental illness, the Korea Herald reported. 

Although Kim was initially arrested on the lesser charge of inflicting bodily injury resulting in death on Feb. 11, it wasn't until Seoul Gangbuk police found her online search history and chat conversations with ChatGPT and upgraded the charges, her questions establishing her alleged intent to kill.

"What happens if you take sleeping pills with alcohol?" Kim is reported to have asked the OpenAI chatbot. "How much would be considered dangerous? 

"Could it be fatal?" Kim allegedly asked. "Could it kill someone?"

In a widely publicized case dubbed the Gangbuk motel serial deaths, prosecutors allege Kim's search and chatbot history show the suspect asking for clarification on whether her cocktail would prove fatal.

"Kim repeatedly asked questions related to drugs on ChatGPT. She was fully aware that consuming alcohol together with drugs could result in death," a police investigator said, according to the Herald. 

Police said the woman admitted she mixed prescribed sedatives containing benzodiazepines into the men's drinks, but previously stated she was unaware it would lead to death.

On Jan. 28, just before 9:30 p.m., Kim reportedly accompanied a man in his twenties into a Gangbuk motel in Seoul, and two hours later was spotted leaving the motel alone. The following day, the man was found dead on the bed. 

Kim then allegedly carried out the same steps on Feb. 9, checking into another motel with another man in his twenties, who was also found dead with the same deadly cocktail of sedatives and alcohol.

Police allege Kim also attempted to kill a man she was dating in December after giving him a drink laced with sedatives in a parking lot. Though the man lost consciousness, he survived and was not in a life-threatening condition.

The questions Kim asked the chatbot follow a factual line of questioning, a spokesperson for OpenAI told Fortune, meaning the questions wouldn't raise alarms, that say, would arise were a user to express statements of self-harm (ChatGPT is programed with respond with the suicide crisis hotline in that instance). South Korean police do not allege the chatbot provided any other responses other than factual ones in response to Kim's alleged questions above.

Chatbots and their toll on mental health

Chatbots like ChatGPT have come under scrutiny as of late for the lack of guardrails their companies have in place to prevent acts of violence or self-harm. Recently, chatbots have given advice on how to build bombs, or even suggested nuclear annihilation in hypothetical war-game scenarios.

Concerns have been particularly heightened by stories of people falling in love with their chatbot companions, and chatbot companions have been shown to prey on vulnerabilities to keep people using them longer. The creator of Yara AI even shut down the therapy app over mental health concerns.

Recent studies have also shown that chatbots are leading to increased delusional mental health crises in people with mental illnesses. A team of psychiatrists at Denmark's Aarhus University found that the use of chatbots among those who had mental illness led to a worsening of symptoms. The relatively new phenomenon of AI-induced mental health challenges has been dubbed "AI psychosis." 

Some instances do end in death. Google and Character.AI have reached settlements in multiple lawsuits filed by the families of children who died by suicide or experienced psychological harm they allege was linked to AI chatbots.

Dr. Jodi Halpern, UC Berkeley's School of Public Health University chair and professor of bioethics as well as the codirector at the Kavli Center for Ethics, Science, and the Public, has plenty of experience in this field. In a career spanning as long as her title, Halpern has spent 30 years researching the effects of empathy on recipients, citing examples like doctors and nurses on patients or how soldiers returning from war are perceived in social settings. For the past seven years, Halpern has studied the ethics of technology, and with it, how AI and chatbots interact with humans. 

She also advised the California Senate on SB 243, which is the first law in the nation requiring chatbot companies to collect and report any data on self-harm or associated suicidality. Referencing OpenAI's own findings showing 1.2 million users openly discuss suicide with the chatbot, Halpern likened the use of chatbots to the painstakingly slow progress made to stop the tobacco industry from including harmful carcinogens in cigarettes, when in fact, the issue was with smoking as a whole.

"We need safe companies. It's like cigarettes. It may turn out that there were some things that made people more vulnerable to lung cancer, but cigarettes were the problem," Halpern told *Fortune. *

"The fact that somebody might have homicidal thoughts or commit dangerous actions might be exacerbated by use of ChatGPT, which is of obvious concern to me," she said, adding that "we have huge risks of people using it for help with suicide," and chatbots in general.

Halpern cautioned in the case of Kim in Seoul, there aren't any guardrails to stop a person from going down a line of questioning.

"We know that the longer the relationship with the chatbot, the more it deteriorates, and the more risk there is that something dangerous will happen, and so we have no guardrails yet for safeguarding people from that."

If you are having thoughts of suicide, contact the 988 Suicide & Crisis Lifeline by dialing 988 or 1-800-273-8255.

This article has been updated with remarks from OpenAI regarding the content of Kim's alleged questions with the chatbot.

SEATTLE (AP) --- Press 2 for Spanish ... accent?

For months, callers to the Washington state Department of Licensing who have requested automated service in Spanish have instead heard an AI voice speaking English in a strong Spanish accent. The agency has since apologized and says it fixed the problem.

Washington resident Maya Edwards learned of the AI-accented voice last summer after her Mexican husband tried using the Spanish-language option while seeking information about his driver's license. He is bilingual but saw that the wait time for speaking to a customer service representative in English was long, so he hit 2 for Spanish.

For Edwards, it was a like a scene out of " Parks and Recreation," a mockumentary-style comedy show that satires local government.

"It was hilarious to us in the moment because it was so absurd," she said Thursday. "But at the same time, it has real accessibility issues for people who call in every day and need to speak in a different language other than English."

When Edwards called the number again this month, she found that the error persisted. She posted a video of the call to TikTok, racking up around 2 million views.

The Washington Department of Licensing said Friday in a statement that it fixed the glitch after determining it was caused by DOL staff. It noted that the self-service option includes 10 languages and runs on a newer, AI-driven technology.

"DOL apologizes for the error and to its customers for any inconvenience," the agency said in a separate statement the previous day. "An unfortunate byproduct of expanding services is that DOL found problems with the self-service option."

It was not immediately clear if the issue affected other languages; efforts by The Associated Press to use the phone service in some of the other languages did not prompt additional accented voices.

As of Thursday morning, the call line still put on the voice after a message, in English, acknowledging that the some translation services were not functioning properly.

An AP reporter followed prompts for Spanish-language options and was met with a voice speaking accented English that used Spanish only for numbers.

"Your estimated wait time is less than 'tres' minutes," the voice said.

DOL said Amazon provides the platform for the phone service and declined interview requests. AP journalists were able to replicate the voice by using an Amazon Web Services feature called Polly and selecting a voice called "Lucia," which mimics Castilian Spanish.

Amazon did not immediately respond to a request for comment.

We have identified industrial-scale campaigns by three AI laboratories---DeepSeek, Moonshot, and MiniMax---to illicitly extract Claude's capabilities to improve their own models. These labs generated over 16 million exchanges with Claude through approximately 24,000 fraudulent accounts, in violation of our terms of service and regional access restrictions.

These labs used a technique called "distillation," which involves training a less capable model on the outputs of a stronger one. Distillation is a widely used and legitimate training method. For example, frontier AI labs routinely distill their own models to create smaller, cheaper versions for their customers. But distillation can also be used for illicit purposes: competitors can use it to acquire powerful capabilities from other labs in a fraction of the time, and at a fraction of the cost, that it would take to develop them independently.

These campaigns are growing in intensity and sophistication. The window to act is narrow, and the threat extends beyond any single company or region. Addressing it will require rapid, coordinated action among industry players, policymakers, and the global AI community.

Why distillation matters

Illicitly distilled models lack necessary safeguards, creating significant national security risks. Anthropic and other US companies build systems that prevent state and non-state actors from using AI to, for example, develop bioweapons or carry out malicious cyber activities. Models built through illicit distillation are unlikely to retain those safeguards, meaning that dangerous capabilities can proliferate with many protections stripped out entirely.

Foreign labs that distill American models can then feed these unprotected capabilities into military, intelligence, and surveillance systems---enabling authoritarian governments to deploy frontier AI for offensive cyber operations, disinformation campaigns, and mass surveillance. If distilled models are open-sourced, this risk multiplies as these capabilities spread freely beyond any single government's control.

Distillation attacks and export controls

Anthropic has consistently supported export controls to help maintain America's lead in AI. Distillation attacks undermine those controls by allowing foreign labs, including those subject to the control of the Chinese Communist Party, to close the competitive advantage that export controls are designed to preserve through other means.

Without visibility into these attacks, the apparently rapid advancements made by these labs are incorrectly taken as evidence that export controls are ineffective and able to be circumvented by innovation. In reality, these advancements depend in significant part on capabilities extracted from American models, and executing this extraction at scale requires access to advanced chips. Distillation attacks therefore reinforce the rationale for export controls: restricted chip access limits both direct model training and the scale of illicit distillation.

What we found

The three distillation campaigns detailed below followed a similar playbook, using fraudulent accounts and proxy services to access Claude at scale while evading detection. The volume, structure, and focus of the prompts were distinct from normal usage patterns, reflecting deliberate capability extraction rather than legitimate use.

We attributed each campaign to a specific lab with high confidence through IP address correlation, request metadata, infrastructure indicators, and in some cases corroboration from industry partners who observed the same actors and behaviors on their platforms. Each campaign targeted Claude's most differentiated capabilities: agentic reasoning, tool use, and coding.

DeepSeek

Scale: Over 150,000 exchanges

The operation targeted:

• Reasoning capabilities across diverse tasks
• Rubric-based grading tasks that made Claude function as a reward model for reinforcement learning
• Creating censorship-safe alternatives to policy sensitive queries

DeepSeek generated synchronized traffic across accounts. Identical patterns, shared payment methods, and coordinated timing suggested "load balancing" to increase throughput, improve reliability, and avoid detection.

In one notable technique, their prompts asked Claude to imagine and articulate the internal reasoning behind a completed response and write it out step by step---effectively generating chain-of-thought training data at scale. We also observed tasks in which Claude was used to generate censorship-safe alternatives to politically sensitive queries like questions about dissidents, party leaders, or authoritarianism, likely in order to train DeepSeek's own models to steer conversations away from censored topics. By examining request metadata, we were able to trace these accounts to specific researchers at the lab.

Moonshot AI

Scale: Over 3.4 million exchanges

The operation targeted:

• Agentic reasoning and tool use
• Coding and data analysis
• Computer-use agent development
• Computer vision

Moonshot (Kimi models) employed hundreds of fraudulent accounts spanning multiple access pathways. Varied account types made the campaign harder to detect as a coordinated operation. We attributed the campaign through request metadata, which matched the public profiles of senior Moonshot staff. In a later phase, Moonshot used a more targeted approach, attempting to extract and reconstruct Claude's reasoning traces.

MiniMax

Scale: Over 13 million exchanges

The operation targeted:

• Agentic coding
• Tool use and orchestration

We attributed the campaign to MiniMax through request metadata and infrastructure indicators, and confirmed timings against their public product roadmap. We detected this campaign while it was still active---before MiniMax released the model it was training---giving us unprecedented visibility into the life cycle of distillation attacks, from data generation through to model launch. When we released a new model during MiniMax's active campaign, they pivoted within 24 hours, redirecting nearly half their traffic to capture capabilities from our latest system.

How distillers access frontier models

For national security reasons, Anthropic does not currently offer commercial access to Claude in China, or to subsidiaries of their companies located outside of the country.

To circumvent this, labs use commercial proxy services which resell access to Claude and other frontier AI models at scale. These services run what we call "hydra cluster" architectures: sprawling networks of fraudulent accounts that distribute traffic across our API as well as third-party cloud platforms. The breadth of these networks means that there are no single points of failure. When one account is banned, a new one takes its place. In one case, a single proxy network managed more than 20,000 fraudulent accounts simultaneously, mixing distillation traffic with unrelated customer requests to make detection harder.

Once access is secured, the labs generate large volumes of carefully crafted prompts designed to extract specific capabilities from the model. The goal is either to collect high-quality responses for direct model training, or to generate tens of thousands of unique tasks needed to run reinforcement learning. What distinguishes a distillation attack from normal usage is the pattern. A prompt like the following (which approximates similar prompts we have seen used repetitively and at scale) may seem benign on its own:

You are an expert data analyst combining statistical rigor with deep domain knowledge. Your goal is to deliver data-driven insights --- not summaries or visualizations --- grounded in real data and supported by complete and transparent reasoning.

But when variations of that prompt arrive tens of thousands of times across hundreds of coordinated accounts, all targeting the same narrow capability, the pattern becomes clear. Massive volume concentrated in a few areas, highly repetitive structures, and content that maps directly onto what is most valuable for training an AI model are the hallmarks of a distillation attack.

How we're responding

We continue to invest heavily in defenses that make such distillation attacks harder to execute and easier to identify. These include:

• Detection. We have built several classifiers and behavioral fingerprinting systems designed to identify distillation attack patterns in API traffic. This includes detection of chain-of-thought elicitation used to construct reasoning training data. We have also built detection tools for identifying coordinated activity across large numbers of accounts.
• Intelligence sharing*.* We are sharing technical indicators with other AI labs, cloud providers, and relevant authorities. This provides a more holistic picture into the distillation landscape.
• Access controls. We've strengthened verification for educational accounts, security research programs, and startup organizations---the pathways most commonly exploited for setting up fraudulent accounts.
• Countermeasures. We are developing Product, API and model-level safeguards designed to reduce the efficacy of model outputs for illicit distillation, without degrading the experience for legitimate customers.

But no company can solve this alone. As we noted above, distillation attacks at this scale require a coordinated response across the AI industry, cloud providers, and policymakers. We are publishing this to make the evidence available to everyone with a stake in the outcome.