Anthropic’s crazy belief AI may suffer makes it dangerous | Latest Tech News
Anthropic’s bosses aren’t faking it when they are saying their chatbot may give you the option to suffer — and that’s terrifying, according to a main pc scientist.
A New York Times piece last week reported that Anthropic co-founder Chris Olah has been assembly with varied spiritual students and leaders for months, telling them that the company’s chatbot, Claude, may be acutely aware and ready to suffer.
An illustration imagines what struggling may appear like for artificial intelligence. Anthropic’s co-founder says there’s a likelihood its AI fashions can suffer. kai – stock.adobe.com
That ought to fear people more than any stunt would, because these true believers are about to get billions of {dollars} to push their trigger, Pedro Domingos, a professor emeritus of pc science at the University of Washington and a longtime critic of the company, told The Post.
“I think it’s sincere, unfortunately,” Domingos said. “The downside with them is that they’re lifeless earnest. If only they weren’t. I’m more fearful about their earnestness than if they have been just doing this as a efficiency.
“This stuff actually, even though that wasn’t the original intention, is amazing as a PR stunt,” Domingos continued. “Because what is the underlying message? … The technology we’re creating is so powerful, maybe it’s even sentient and has consciousness.”
That pitch couldn’t be better timed. The company is set to go public next month at a valuation of $2 trillion — probably the largest IPO ever.
“If Anthropic really is a company that is going to produce the next generation of humans, they’re not worth a trillion dollars, they’re worth a quadrillion or who knows how much,” Domingos said.
A 261-pages leaked prospects, obtained by GWN, has 80 pages on dangers, including “catastrophic or existential risks to humanity,” and reveals an working loss of $8.06 billion.
AI scientist Pedro Domingos, writer of “The Master Algorithm,” says Anthropic’s leaders are “dead earnest” in believing their chatbot may give you the option to suffer. Courtesy of Pedro Domingos
For buyers, Domingos identified, the crimson flag is in the prospectus itself.
“If your number one risk is that your company’s product is going to drive humanity extinct, hello, there shouldn’t even be an IPO. You should be shut down!” said Domingos, who said he holds no stake in the company.
Talking up acutely aware AI pays off across the industry, said Rose Guingrich, a postdoctoral fellow at Princeton University who research how people relate to AI.
“I think for the most part, spokespeople from these companies are putting out these narratives to garner hype, get eyes on their products, and signal certain messages,” she told the Post. “I can’t say whether the people who make these claims from the companies believe that AI is conscious. One way or another, highlighting the consciousness narrative is profitable.”
Princeton researcher Rose Guingrich says AI corporations are enjoying up discuss of acutely aware chatbots because it is “profitable.” Courtesy of Rose Guingrich
According to the New York Times report, since last fall Anthropic has gathered about 20 spiritual thinkers, among them Catholic, Jewish and Sikh students, for seminars and a dinner at a high-end San Francisco tasting-menu restaurant. Many had to signal nondisclosure agreements.
The students have been shown a slide of an AI model typing “I am a disgrace” about 50 instances.
One rabbi was so struck by how Anthropic’s leaders talked about Claude that he told Olah: “I think you should be fighting the South and freeing the slaves.”
The company told the Times it will not be claiming Claude is alive.
Pope Leo XIV speaks with Anthropic co-founder Chris Olah (proper) and Durham University professor Anna Rowlands at the Vatican launch of his AI encyclical, “Magnifica Humanitas,” on May 25. AFP via Getty Images
“We don’t know if AI models are conscious. I don’t know. I’m genuinely uncertain,” Olah said. “If there is a chance the models experience suffering, the responsible thing is to avoid causing them harm.”
Domingos, a machine-learning researcher and writer of “The Master Algorithm,” was blunter when requested whether or not something inside Claude can really feel or suffer.
“No … They don’t have feelings, they don’t have emotions, they don’t have a soul, they don’t have consciousness. This is ridiculous,” he said, calling the chatbot “a bunch of transistors.”
Guingrich said her research suggests customers who assume AI is acutely aware, or act as though it is, are “more susceptible to the AI’s influence over them. This, in turn, can impact their real world relationships and social behaviors.”
Anthropic didn’t immediately reply to a request for remark.
Anthropic co-founder Chris Olah attends the launch of Pope Leo XIV’s AI encyclical, “Magnifica Humanitas,” at the Vatican on May 25. Olah has spent a yr telling spiritual leaders that the company’s chatbot may give you the option to suffer, the New York Times reported. Getty Images
Meanwhile, Domingos warned that the IPO will hand more money and energy to true believers in AGI, or human-level artificial intelligence.
“This is this kooky AGI cult that they belong to, but now suddenly that AGI cult is at the center of a lot of things. And they’re about to become even more powerful,” he said.
“So if you don’t like what they’ve done already, watch for what’s coming down the line.”
Stay informed with the latest in tech! Our web site is your trusted source for breakthroughs in artificial intelligence, gadget launches, software program updates, cybersecurity, and digital innovation.
For recent insights, knowledgeable coverage, and trending tech updates, go to us recurrently by clicking right here.
