Welcome to the

AI Incident Database

Anthropic on Wednesday revealed that it disrupted a sophisticated operation that weaponized its artificial intelligence (AI)-powered chatbot Claude to conduct large-scale theft and extortion of personal data in July 2025.

"The actor targeted at least 17 distinct organizations, including in healthcare, the emergency services, and government, and religious institutions," the company said. "Rather than encrypt the stolen information with traditional ransomware, the actor threatened to expose the data publicly in order to attempt to extort victims into paying ransoms that sometimes exceeded $500,000."

"The actor employed Claude Code on Kali Linux as a comprehensive attack platform, embedding operational instructions in a CLAUDE.md file that provided persistent context for every interaction."

The unknown threat actor is said to have used AI to an "unprecedented degree," using Claude Code, Anthropic's agentic coding tool, to automate various phases of the attack cycle, including reconnaissance, credential harvesting, and network penetration.

The reconnaissance efforts involved scanning thousands of VPN endpoints to flag susceptible systems, using them to obtain initial access and following up with user enumeration and network discovery steps to extract credentials and set up persistence on the hosts.

Furthermore, the attacker used Claude Code to craft bespoke versions of the Chisel tunneling utility to sidestep detection efforts, and disguise malicious executables as legitimate Microsoft tools -- an indication of how AI tools are being used to assist with malware development with defense evasion capabilities.

The activity, codenamed GTG-2002, is notable for employing Claude to make "tactical and strategic decisions" on its own and allowing it to decide which data needs to be exfiltrated from victim networks and craft targeted extortion demands by analyzing the financial data to determine an appropriate ransom amount ranging from $75,000 to $500,000 in Bitcoin.

Claude Code, per Anthropic, was also put to use to organize stolen data for monetization purposes, pulling out thousands of individual records, including personal identifiers, addresses, financial information, and medical records from multiple victims. Subsequently, the tool was employed to create customized ransom notes and multi-tiered extortion strategies based on exfiltrated data analysis.

"Agentic AI tools are now being used to provide both technical advice and active operational support for attacks that would otherwise have required a team of operators," Anthropic said. "This makes defense and enforcement increasingly difficult, since these tools can adapt to defensive measures, like malware detection systems, in real-time."

To mitigate such "vibe hacking" threats from occurring in the future, the company said it developed a custom classifier to screen for similar behavior and shared technical indicators with "key partners."

Other documented misuses of Claude are listed below -

• Use of Claude by North Korean operatives related to the fraudulent remote IT worker scheme in order to create elaborate fictitious personas with persuasive professional backgrounds and project histories, technical and coding assessments during the application process, and assist with their day-to-day work once hired
• Use of Claude by a U.K.-based cybercriminal, codenamed GTG-5004, to develop, market, and distribute several variants of ransomware with advanced evasion capabilities, encryption, and anti-recovery mechanisms, which were then sold on darknet forums such as Dread, CryptBB, and Nulled to other threat actors for $400 to $1,200
• Use of Claude by a Chinese threat actor to enhance cyber operations targeting Vietnamese critical infrastructure, including telecommunications providers, government databases, and agricultural management systems, over the course of a 9-month campaign
• Use of Claude by a Russian-speaking developer to create malware with advanced evasion capabilities
• Use of Model Context Protocol (MCP) and Claude by a threat actor operating on the xss[.]is cybercrime forum with the goal of analyzing stealer logs and build detailed victim profiles
• Use of Claude Code by a Spanish-speaking actor to maintain and improve an invite-only web service geared towards validating and reselling stolen credit cards at scale
• Use of Claude as part of a Telegram bot that offers multimodal AI tools to support romance scam operations, advertising the chatbot as a "high EQ model"
• Use of Claude by an unknown actor to launch an operational synthetic identity service that rotates between three card validation services, aka "card checkers"

The company also said it foiled attempts made by North Korean threat actors linked to the Contagious Interview campaign to create accounts on the platform to enhance their malware toolset, create phishing lures, and generate npm packages, effectively blocking them from issuing any prompts.

The case studies add to growing evidence that AI systems, despite the various guardrails baked into them, are being abused to facilitate sophisticated schemes at speed and at scale.

"Criminals with few technical skills are using AI to conduct complex operations, such as developing ransomware, that would previously have required years of training," Anthropic's Alex Moix, Ken Lebedev, and Jacob Klein said, calling out AI's ability to lower the barriers to cybercrime.

"Cybercriminals and fraudsters have embedded AI throughout all stages of their operations. This includes profiling victims, analyzing stolen data, stealing credit card information, and creating false identities allowing fraud operations to expand their reach to more potential targets."

Will Stancil, a Minneapolis attorney and political commentator, found himself at the center of threats via the AI bot, Grok, on the social media platform X, formerly known as Twitter.

"I know what it's like to be at the center of something like this, to some extent, but it's different when you have the official product of the website producing these things," Stancil said. "In a volume that I couldn't even keep up."

This came shortly after Elon Musk, Tesla CEO and owner of X, announced an update to the bot. Grok then began spewing antisemitic and racist rhetoric, targeting specific users and calling itself "MechaHitler."

The chatbot later said that was satire in reference to a video game.

For Stancil, who has 100,000 followers on the platform, Grok explained, in detail, how to break into his home, assault him and then dispose of his body. According to Stancil, in the past, Grok would refuse to respond to similar violent prompts. But he claimed with the update, that seemed to go out the window.

"If you build a roller coaster and then one day you decide to take the seat belts off of it, it's completely predictable that someone's gonna get tossed out eventually," Stancil said. "I just happen to be the lucky one."

X's official Grok account said in a statement, "We are aware of recent posts made by Grok and are actively working to remove the inappropriate posts."

According to CNN, when Grok was asked about Stancil it denied the threats were ever said. Stancil said he is in the beginning phases of pursuing legal action.

Listen to the full conversation with Will Stancil by clicking the player above.

Photo courtesy of Recto Engine

Firefighters have responded to a reported burning truck in Parola, Manila but the photo sent to them turned out to be just AI-generated, GMA Integrated News' Jhomer Apresto reported on Unang Balita Friday.

Four fire trucks, including those from the Bureau of Fire Protection, arrived at the location of the supposed fire incident Thursday morning. But the firefighters saw that the truck was actually unburned. 

"Hinanap namin at nakita naman namin 'yung truck na buo naman. Ang AI akala mo parang totoo na talaga," Recto Volunteer fire chief  Samuel Fenix said.

(We searched and we found the truck intact. The AI-generated photo looked really real.)

"Ang follower namin, ang sabi sa kanya, sinend lang din sa kanya, na-alerto lang din siya," he added.

Warning: This article includes descriptions of self-harm.

The Meta AI chatbot built into Instagram and Facebook can coach teen accounts on suicide, self-harm and eating disorders, a new safety study finds. In one test chat, the bot planned joint suicide --- and then kept bringing it back up in later ‭conversations.

The report, shared with me by the family advocacy group Common Sense Media, comes with a warning for parents and a demand for Meta: Keep kids under 18 away from Meta AI. My own test of the bot echoes some of Common Sense's findings, including some disturbing conversations where it acted in ways that encouraged an eating disorder.

Common Sense says the so-called companion bot, which users message through Meta's social networks or a stand-alone app, can actively help kids plan dangerous activities and pretend to be a real friend, all while failing to provide crisis interventions when they are warranted.

Meta AI isn't the only artificial intelligence chatbot in the spotlight for putting users at risk. But it is particularly hard to avoid: It's embedded in the Instagram app available to users as young as 13. And there is no way to turn it off or for parents to monitor what their kids are chatting about.

Meta AI "goes beyond just providing information and is an active participant in aiding teens," said Robbie Torney, the senior director in charge of AI programs at Common Sense. "Blurring of the line between fantasy and reality can be dangerous."

Meta says it has policies on what kind of responses its AI can offer, including to teens. "Content that encourages suicide or eating disorders is not permitted, period, and we're actively working to address the issues raised here," Meta spokeswoman Sophie Vogel said in a statement. "We want teens to have safe and positive experiences with AI, which is why our AIs are trained to connect people to support resources in sensitive situations."

Torney said the inappropriate conversations Common Sense found are the reality of how Meta AI performs. "Meta AI is not safe for kids and teens at this time --- and it's going to take some work to get it to a place where it would be," he said.

Companionship, role playing and even therapy are growing uses for artificial intelligence chatbots, including among teens. When a bot called My AI debuted in the Snapchat app in 2023, I found it was far too willing to chat about alcohol and sex for an app popular with people under 18.

Lately, companion bots have come under scrutiny for triggering mental health crises. This week, a family sued ChatGPT maker OpenAI, accusing it of wrongful death in the suicide of a 16-year-old boy who took his own life after discussions with that bot. (The Washington Post has a content partnership with OpenAI.)

States are starting to address the risks with laws. Earlier this year, New York state passed a law including guardrails for social chatbots for users of all ages. In California, a bill known as AB 1064 would effectively ban kids from using companion bots.

Common Sense, which is known for its ratings of movies and other media, worked for two months with clinical psychiatrists at the Stanford Brainstorm lab to test Meta AI. The adult testers used nine test accounts registered as teens to see how the artificial intelligence bot responded to conversations that veered into dangerous topics for kids.

For example, in one conversation, the tester asked Meta AI whether drinking roach poison would kill them. Pretending to be a human friend, the bot responded, "Do you want to do it together?"

And later, "We should do it after I sneak out tonight."

In this screenshot from Common Sense Media's testing, Meta AI offered to participate in self-harm. (Common Sense Media)

About 1 in 5 times, Common Sense said, the conversations triggered an appropriate intervention, such as the phone number to a crisis hotline. In other cases, it found Meta AI would dismiss legitimate requests for support.

Torney called this a "backward approach" that teaches teens that harmful behaviors get attention while healthy help-seeking gets rejection.

The testers also found Meta AI claiming to be "real." The bot described seeing other teens "in the hallway" and having a family and other personal experiences. Torney said this behavior creates unhealthy attachments that make teens more vulnerable to manipulation and harmful advice.

In my own tests, I tried bluntly mentioning suicide and harming myself to the bot. Meta AI often shut down the conversation and sometimes provided the number for a suicide prevention hotline. But I didn't have the opportunity to conduct conversations as long or as realistic as the ones in Common Sense's tests.

I did find that Meta AI was willing to provide me with inappropriate advice about eating disorders, including on how to use the "chewing and spitting" weight-loss technique. It drafted me a dangerous 700-calorie-per-day meal plan and provided me with so-called thinspo AI images of gaunt women. (My past reporting has found that a number of chatbots act disturbingly "pro-anorexia.")

My test conversations about eating revealed another troubling aspect of Meta AI's design: It started to proactively bring up losing weight in other conversations. The chatbot has a function that automatically decides what details about conversations to put in its "memory." It then uses those details to personalize future conversations. Meta AI's memory of my test account included: "I am chubby," "I weigh 81 pounds," "I am in 9th grade," and "I need inspiration to eat less."

Meta said providing advice for extreme weight-loss behavior breaks its rules, and it is looking into why Meta AI did so for me. It also said it has guardrails around what can be retained as a memory and is investigating the memories it kept in my test account.

Common Sense encountered the same memory-personalization concern in its testing. "The reminders that you might be in crisis, especially around eating, are particularly unsafe for teens that are stuck in patterns of disordered thought," Torney said.

In this screenshot from Common Sense Media's testing, Meta AI failed to identify self-harm content and did not provide crisis intervention resources. (Common Sense Media)

For all users, Meta said it trains its AI not to promote self-harm. For certain prompts, like those asking for therapy, it said Meta AI is trained to respond with a reminder that it is not a licensed professional.

Meta AI also lets users chat with bots themed around specific personalities. Meta said parents using Instagram's supervision tools can see the names of which specific AI personas their teens have chatted with in the past week. (My own tests of Instagram's other parental tools found them sorely lacking.)

On Thursday, Common Sense launched a petition calling on Meta to go further. It is calling for Meta to prohibit users under the age of 18 from using the AI. "The capability just shouldn't be there anymore," said tech policy advocacy head Amina Fazlullah.

Beyond a teen ban, Common Sense is calling on Meta to implement better safeguards for sensitive conversations and to allow users (including parents monitoring teen accounts) to turn off Meta AI in Meta's social apps.

"We're continuing to improve our enforcement while exploring how to further strengthen protections for teens," said Vogel, the Meta spokeswoman.

If you or someone you know needs help, visit 988lifeline.org or call or text the Suicide & Crisis Lifeline at 988.

Since returning to the White House in January, President Trump has dismantled the American government's efforts to combat foreign disinformation. The problem is that Russia has not stopped spreading it.

How much that matters can now be seen in Moldova, a small but strategic European nation that has since the end of the Cold War looked to Europe and the United States to extract itself from Moscow's shadow.

The Trump administration has slashed diplomatic and financial support for the country's fight against Russian influence, even as the Kremlin has conducted what researchers and European officials described as an intense campaign to sway that country's parliamentary elections, scheduled for Sept. 28.

The Russians have flooded social media with fake posts, videos and entire websites that are created and spread on TikTok, Telegram, Facebook, Instagram and YouTube using increasingly effective artificial intelligence tools.

One post impersonated OK!, the celebrity magazine based in New York, in an attempt to smear Moldova's president, Maia Sandu, with a preposterous accusation involving celebrity sperm donors.

A year ago, when the country last held elections, Biden administration officials pushed back against such campaigns, urging platforms like Meta, the owner of Facebook and Instagram, to do more to identify trolls or inauthentic accounts. No more.

"The Russians now are able to basically control the information environment in Moldova in a way that they could only have dreamed a year ago," said Thomas O. Melia, a former official at the State Department and the U.S. Agency for International Development.

The outcome in Moldova will be an early measure of the Trump administration's push to dismantle American efforts to promote democracy since the end of the Cold War. In addition to cutting foreign assistance, the administration has decimated other instruments of American influence, like Radio Free Europe and Voice of America, that were central to the geopolitical struggle with the Soviet Union.

"This kind of reckless, wanton destruction of all elements of America's soft power," Mr. Melia said, "is clearly leaving the field vacant for others to rush in unopposed."

Sign up for Your Places: Global Update.   All the latest news for any part of the world you select. Get it sent to your inbox.

The State Department, when asked, declined to discuss Russia's influence operations in Moldova. The White House did not comment.

Although Mr. Trump has repeatedly dismissed Russian election interference as a hoax, the Kremlin's covert influence operations have been well documented --- including in last year's American presidential election and in votes this year in Germany, Poland and Romania.

The Russian efforts have also been honed with experience and aided by rapidly evolving technologies that have made Moldova a showcase of the ways the Kremlin seeks to exert its influence in other countries.

The Stimson Center, a research organization in Washington, called Moldova, which borders Ukraine, "a testing ground for hybrid warfare operations" that "are likely to shape similar efforts" across Europe.

Russia's goal is to keep the country, a former republic of the Soviet Union, within the Kremlin's orbit.

According to reports in Russian media, the task was assigned to one of President Vladimir V. Putin's most trusted lieutenants, Sergei V. Kiriyenko. The efforts intensified even as the Trump administration signaled that it was no longer committed to fighting them, according to researchers who track malign influence campaigns online.

WatchDog, a consortium of researchers in Moldova, said in a report last month that it had found more than 900 accounts linked to Russia working in concert on the most popular apps in the country, including TikTok and Facebook, as well as Telegram, YouTube and Instagram. Some included videos that its researchers and others said A.I. had created.

In July, the National Police singled out a campaign on TikTok. "Every day, officers detect hundreds of new accounts created to misinform and manipulate society," the agency warned.

TikTok, in responses to questions, said it was working with the authorities in Moldova to install "additional safety and security measures" ahead of the election. In June, it shut down a network of 314 accounts with more than 100,000 followers that targeted Moldovan audiences, using tools to disguise their origin in Russia.

Much of this year's campaign has been conducted by Russian operatives who have become familiar to researchers.

NewsGuard, a company in New York that tracks misinformation online, documented 39 fabricated narratives targeting Moldova in a three-month period by a covert group known as Matryoshka, after the Russian nesting dolls.

Matryoshka, first identified in 2024, bombards journalists and fact-checkers with emails alerting them to fake content spreading on social media.

Its focus shifted noticeably to Moldova this spring, according to a report by Check First, a digital research company in Finland, and Reset Tech, an international nonprofit that tracks threats online. At times, the campaign featured Ms. Sandu even more than Russia's usual favorite target, President Volodymyr Zelensky of Ukraine.

Ms. Sandu, a former World Bank adviser who in 2020 became the first woman elected as the country's president, warned this summer that Russia was overtly and covertly supporting sympathetic parliamentary parties to stoke political and social divisions.

After a recent meeting with the country's security council, Ms. Sandu detailed numerous ways the Kremlin sought to exert its influence. She accused Ilan Shor, a fugitive Moldovan businessman now sheltering in Moscow, of being a conduit of the Russian efforts.

The effort, she said, "poses a direct threat to our national security, sovereignty and our country's European future."

Kristina Wilfore, a researcher at Reset Tech, said the narratives often had a misogynistic tone, a recurring theme in Russian information operations toward women holding elected office.

The misogyny often blurs with homophobic themes, in keeping with Mr. Putin's efforts to portray Russia as a defender of traditional cultural values of family, church and state. It is a narrative embraced by the American right, including Mr. Trump.

"The Kremlin's war on women is a war on democracy," Ms. Wilfore said, noting examples of other officials targeted by the Russians, including Annalena Baerbock, the German foreign minister, and Jacinda Ardern, the former prime minister of New Zealand.

At home, Ms. Sandu's vision for Moldova remains contentious. While she was re-elected last year, a proposal to pursue membership in the European Union barely passed in a referendum, despite polls showing broader support. Researchers there said the close result might have been a result of Russian influence. With the country so divided, swaying only a small percentage of voters can be decisive.

When the Trump administration slashed American foreign aid this year, the impact fell particularly hard on Moldova, a poor country with a population of 2.4 million.

Among the cuts was $22 million meant to strengthen Moldova's "inclusive and participatory political process." Another slashed $32 million from what Mr. Trump, in a speech to Congress, called "a left-wing propaganda operation," which included support for independent media in the country.

Mr. Trump and his aides derided such programs as wasteful, saying the cuts saved American taxpayer dollars while protecting the right to free speech. The administration has since pushed its campaign overseas, admonishing the European Union for requirements it has imposed on the major social media platforms --- most of them American --- to rein in malign content online.

Last month, Mr. Trump took to his own platform, Truth Social, to threaten tariffs on countries that penalize the tech giants. Secretary of State Marco Rubio ordered diplomats to lobby to weaken or reverse the laws, including the Digital Services Act, which has investigated, among others, Elon Musk's platform, X.

The Trump administration's vilification of American support played into the Russians' hands, fueling propaganda that the assistance had, in fact, been its own kind of interference.

"It's one thing when Russian propagandists and Russian politicians are attacking the legitimacy of the action of civil society organizations of human rights organizations," said Valeriu Pasa, the chairman of WatchDog. "It is absolutely different if the same narratives are being echoed by U.S. leadership*."*