Anthropic, the self-avowed moral center of the AI industry, has been caught spying on its users.
As Ars Technica reports, a security researcher last week uncovered spyware-like code in the company’s Claude Code AI model designed to collect data on Chinese users without detection.
The researcher, known by the pseudonym “Thereallo,” found that the code was hidden in the AI’s system prompt, allowing it track a user’s system timezone and usage of a proxy server in order to suss out if they were connected to specific Chinese AI labs.
Anthropic’s explanation for this huge breach in user trust left much to be desired. On X, Anthropic engineer Thariq Shihipa wrote that the tracker was added as an “experiment” in March “to prevent account abuse from unauthorized resellers and protect against distillation,” and was supposed to be removed.
“We’ve actually been meaning to take this down for a while,” he offered.
Distillation is the process of training a weaker, “student” model on the outputs of a more advanced “teacher” model. It’s a routine practice in the industry, but major AI developers increasingly feel it’s being abused by upstarts trying to ride their coattails. Earlier this year, Anthropic accused the Chinese AI firms DeepSeek, Moonshot, and MiniMax of illegally distilling its models (an ironic tantrum, given how Anthropic trained its tech in the first place: by scanning and shredding millions of copyrighted books, as well as essentially the entire internet, without permission.) Recent reporting from The Washington Post also exposed that some Chinese resellers are selling access to Pro Claude subscriptions that cost more than $100 a month in the US for about $12 a month.
It’s a genuine issue for Anthropic, but it may have stepped on a landmine by trying to surreptitiously crack down on it. Part of why it earns the loyalty of customers is its much-avowed commitment to ethical and transparent AI development. Scores of ChatGPT users flocked to use Claude when Anthropic took a much publicized stand against the Pentagon by demanding its tech not be used in the mass surveillance of US citizens.
In this case, the data collected wasn’t egregiously invasive — but in principle, a line has been crossed.
“Coding agents already live on the wrong side of a scary boundary,” Thereallo wrote in their post about the findings. “They can inspect code, summarize secrets by accident, run commands, install packages, edit files, and push commits on your local machine.”
But “hiding the signal in the system prompt makes every other privacy claim harder to believe,” they added.
“Companies can protect their models,” they made clear. But “when a tool with filesystem and shell access starts hiding classification bits inside invisible…
Source link
Disclaimer
We strive to uphold the highest ethical standards in all of our reporting and coverage. We blogs.grocliq.com want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It’s possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.
Website Upgradation is going on for any glitch kindly connect at [email protected]
