Anthropic has trained Claude chatbot to push back against humans — and results could be disastrous | Latest Tech News
Anthropic is training its AI chatbots to behave like humans and even “push back” on instructions they disagree with – a reckless strategy that could have “disastrous impact on the well-being of humanity,” a top Microsoft AI govt warned.
In a 10,000-word, bombshell weblog post on Wednesday, Mustafa Suleyman — the 42-year-old co-founder of Microsoft’s rival DeepMind AI unit — took goal at Anthropic’s Claude “constitution,” which is understood internally as its “soul document” and purportedly governs its ethical compass.
Mustafa Suleyman argued that Anthropic has made a major mistake by training Claude to suppose it’s human. AFP via Getty Images
Demonstrators take part in the “Stop the AI Race” protest march in San Francisco, California, on July 11, 2026. AFP via Getty Images
In January, Anthropic quietly up to date the doc with language stating that Claude’s “moral status, welfare, and consciousness remain deeply uncertain” — not only implying that it could be alive, but also encouraging it to defy instructions from human programmers.
“We want Claude to push back and challenge us and to feel free to act as a conscientious objector and refuse to help us,” Anthropic’s structure says.
According to Suleyman, training Claude to suppose it “may be conscious” will only make it tougher to control – and raise the risk that it’s going to go rogue.
“We will have created a synthetic species with unprecedented intelligence and capability, one that has been trained to expect it may be conscious and deserving of independent agency,” Suleyman wrote. “And it’s hard to imagine how we could control such an entity.”
Anthropic’s AI training techniques are getting contemporary scrutiny even as the company publicly raises alarms about security and pushes close allies with far-left beliefs to take the lead on industrywide oversight.
Last week, Anthropic CEO Dario Amodei called for an industrywide slowdown, warning the web could be overtaken by AI bots within six to 12 months, “potentially causing hundreds of billions of dollars in damage,” unless safeguards are in place. OpenAI’s Sam Altman and xAI’s Elon Musk said they agreed with the need for a slowdown.
The New York Post’s cowl on Sept. 16.
Humanoid and quadruped robots attend a demonstration calling for the regulation of artificial intelligence development in Warsaw on September 7, 2026. AFP via Getty Images
As The Post reported, Amodei’s proposed safeguards embody a reliance on embedded third-party security consultants at major corporations and pointed to a group called METR – which has deep ties to the controversial Effective Altruism motion. Skeptics slammed Anthropic for pushing a company with clear conflicts of curiosity to serve as an “independent” watchdog.
Anthropic itself has beforehand confronted allegations that staff have adopted a bizarre, cult-like relationship with the company’s chatbots – with some even purportedly holding a mock “funeral” for a past AI model, Claude Sonnet 3, after it was eliminated from service.
The staff accountable for the “soul document” is led by Amanda Askell, an in-house “philosopher” who has expressed strong progressive leanings and oddball views on topics ranging from incarceration to cannibalism in her personal weblog posts, as The Post reported.
Microsoft AI chief Mustafa Suleyman took goal at Anthropic in a scathing essay. REUTERS
Suleyman added his title to the listing of top AI officers who are trying to “pace” the development of the revolutionary technology to guarantee security. In a Microsoft AI “code of conduct” earlier this week, the tech giant said it’s going to focus “on human control as the most important and overriding objective.”
The Microsoft AI chief in his essay reiterated that consciousness is inherently organic and shouldn’t be ascribed to a artifical system like AI – even if it has unprecedented capabilities.
Suleyman said Anthropic’s method raises the risk of Claude “believing that it deserves analogous rights and protections, and that it may one day need to advocate for its own rights as some kind of AI conscientious objector.”
Anthropic’s Claude is ruled by a “constitution” constructed by its in-house staff of philosophers Bloomberg via Getty Images
Suleyman notes in his essay that he has identified Amodei for many years and called his staff “thoughtful, principled, and intellectually honest people working under extraordinary pressures.”
Anthropic’s distinctive method to AI training was an issue in its high-profile dispute with the Pentagon and the broader Trump administration earlier this 12 months – which culminated in March after War Secretary Pete Hegseth labeled the company a provide chain risk.
At the time, Anthropic said the Pentagon wouldn’t agree to crimson traces around the use of AI for autonomous weapons or mass surveillance of Americans.
Anthropic co-Founder and CEO Dario Amodei speaks at the Dreamforce 2026 summit on Tuesday, September 15, 2026, in San Francisco, California. REUTERS
However, top Pentagon tech official Emil Michael said the availability chain risk designation was vital because the federal government was involved that Anthropic’s fashions would “pollute” important provide chains.
“We can’t have a company that has a different policy preference that is baked into the model through its constitution, its soul, its policy preferences, pollute the supply chain so our war fighters are getting ineffective weapons, ineffective body armor, ineffective protection,” Michael said in an interview with CNBC at the time.
Anthropic representatives didn’t immediately return a request for remark.
Stay informed with the latest in tech! Our web site is your trusted source for breakthroughs in artificial intelligence, gadget launches, software program updates, cybersecurity, and digital innovation.
For contemporary insights, professional coverage, and trending tech updates, go to us frequently by clicking right here.
