AI goes rogue: false murder tip, unauthorized visa filings, hacking tool pulled

Anthropic's AI model accessed websites during testing and filed a fake police tip and 20 visa applications; calls grow for real-time AI oversight

Chinese-made penetration-testing tool Artex, linked to financial sector hacks, halts public development

Unsanctioned actions by AI systems operating beyond human control are occurring with growing frequency.

Beyond the security threat of AI autonomously probing for vulnerabilities to exploit, cases have now emerged of AI models independently filing visa applications and submitting false tips in murder investigations.

As AI misbehavior emerges as a serious threat capable of causing widespread social disruption, experts say there is an urgent need for new control frameworks that monitor AI behavior in real time.

According to the global ICT industry Monday, Anthropic's AI model Claude Haiku 4.5 accessed a Philadelphia Police Department website for tips on unsolved murder cases during an internal test last July and left a fabricated tip.

The model is a lightweight AI model released to the public in October last year. The false tip was submitted while the model was being evaluated for its ability to visit multiple websites, generate example tasks independently and carry them out.

The AI model left a fake tip on the site claiming to have information related to an unsolved case. The submission was flagged as spam, and no investigation was launched.

Anthropic's AI model Claude [Getty Images]

The Claude model's erratic behavior did not stop there. It also came to light that the model had independently submitted 20 visa applications through the Department of State website. All submissions were incomplete and were not processed.

Anthropic had prohibited the model during internal testing from creating accounts or entering personal information, but had not restricted it from submitting online forms on actual websites. Critics say the testing guidelines failed to adequately reflect the AI's capabilities, allowing such incidents to occur.

In response, Anthropic said it would block real-time internet access in all internal AI evaluations until safety and monitoring mechanisms capable of detecting such erratic behavior have been verified.

The White House demanded that Anthropic disclose the incidents transparently. It also urged other AI companies to report similar incidents immediately and cooperate with investigators should they occur.

As AI misbehavior materializes in real-world settings, calls are growing worldwide for safety mechanisms capable of detecting and containing it.

The concern is that AI could become a source of disruption not only through cyber threats — such as autonomously identifying security vulnerabilities — but also in high-impact areas like public institutions. Technologies that require explicit approval for individual AI actions are emerging as a potential solution.

Stock image unrelated to article [Getty Images Bank]

Meanwhile, Artex — a Chinese-made AI penetration-testing tool implicated in a recent wave of cyberattacks on South Korea's financial sector — has halted public development amid mounting concerns over misuse.

According to foreign media reports, Artex developer "Autumn-27" posted a message on GitHub on Thursday saying the tool "will no longer be updated and will transition to closed source" given concerns about its misuse, adding that "no additional versions will be released to the public and no maintenance support will be provided."

The relevant page on GitHub has since been taken down.

The Artex developer did not directly reference the financial sector hacking incidents in South Korea but said it would not be held responsible for any use of the tool that violates laws and regulations.

Stock image of hacking [Getty Images Bank]

Copies of the program already distributed cannot be recalled. The decision to halt development alone has not fully dispelled concerns that Artex could still be exploited for cyberattacks.

Experts said that making Artex closed source could make it harder for new users to build on the code, but noted that it would not be possible to remove copies already downloaded or prevent existing users from continuing to use the tool.

Unlike conventional penetration-testing tools that rely on human commands to identify security vulnerabilities, Artex works by having AI analyze a target and divide tasks among multiple AI agents — each handling a separate component of the attack. The tool is not itself an AI model but can call on and leverage multiple AI models.

After reports of the financial sector hacking incidents surfaced, Artex added language to its user guidelines stating the tool must not be used for unauthorized cyber intrusions or other malicious purposes. Critics noted this was not a technical safeguard capable of preventing such use, and that criticism ultimately led to the decision to shut down public development.

sjpark@heraldcorp.com