Meet Anthropic CEO Dario Amodeis handpicked super-woke globalists he thinks will save us from an AI apocalypse | Latest Tech News
Worried what AI will do to the world? Don’t fret — we acquired a group of woke elitists who will shield us.
That’s the pitch from Anthropic’s CEO Dario Amodei, who says he has discovered an exterior group that can regulate AI and make sure it doesn’t destroy the world.
It’s a group of globalists who imagine in open borders, genderless pronoun and promote “Effective Altruism,” that imagine all the world’s issues may be solved with technology.
In a bombshell essay titled “We Must Pace the Frontier,” Amodei argued that main AI companies ought to commit to permitting “embedded third-party evaluators” from a nonprofit called Model Evaluation and Threat Research, or METR, that would keep “employee-like access” to oversee security efforts and guarantee superior AI fashions don’t flip Terminator and wipe out humanity.
The would-be John Connors at METR purport to be absolutely unbiased, but the group traces its roots instantly back to key figures in Effective Altruism, whose Silicon Valley-based devotees once included disgraced crypto boss Sam Bankman-Fried, one of Anthropic’s greatest early traders.
Anthropic co-founder and CEO Dario Amodei argues for third-party AI reviewers from a nonprofit called Model Evaluation and Threat Research. REUTERS
Effective Altruism, a philosophy that’s grow to be in style with Silicon Valley masters of the universe, advocates for utilizing purpose — and fortunes amassed by tech’s best and brightest — to do the most good for humanity over the long time period.
In the past, it has been used to justify making as a lot money as doable with the thought that you’ll be able to donate more later to do more good – though the motion suffered a big black eye when one of its most well-known proponents, Bankman-Fried, noticed his crypto empire collapse amid fraud costs that despatched him to prison.
Deep EA ties
The overlap between METR and Anthropic is rife with potential conflicts of curiosity, according to critics.
METR spun off from an Effective Altruism-aligned tech incubator called Alignment Research Center (ARC), which is headed by AI researcher Paul Christiano. He was Amodei’s housemate, coworker and research collaborator in the 2010s when they have been both at OpenAI — Anthropic’s greatest rival.
Ajeya Cotra is a member of technical employees at METR. METR
Christiano also served as one of the first 5 trustees of Anthropic’s Long-Term Benefit Trust.
One staffer on METR is Ajeya Cotra, who is married to Christiano and has been a major organizer in the EA world.
The founder and CEO of the group is Beth Barnes, who was employed by Christiano and was concerned in the EA motion at faculty.
Chief scientist Hjalmar Wijk and staffers Megan Kinniment and Pip Arnott all got here from the same research cluster at the University of Oxford — a major feeder for EA.
When reached by cellphone, a METR official acknowledged that the nonprofit has a lot of overlap with Effective Altruism, but insisted its workers have a various array of ideological viewpoints. They’re required to disclose conflicts of curiosity, such as workers holding equity in companies they’re tapped to examine.
“We don’t think any small group should have authority over what happens with AI, and our goal is to get information from inside these companies into the public domain – external investigators can help bring more information to light,” a METR spokesperson said in a assertion.
Beth Barnes is the CEO of METR. METR
“Our funders have no say in the projects we work on and we do not accept any funding from AI companies or their employees.”
Amodei, who has pledged to give away 80% of his personal wealth, has deep ties to EA. His sister Daniela married Holden Karnofsky, the founder of two major EA nonprofits.
METR’s involvement is probably going to be a nonstarter for the Trump administration, which has had a testy relationship with Anthropic over the past yr and has also expressed heavy skepticism of so-called AI “doomers” who argue the technology is on the precipice of catastrophe.
“METR is essentially Anthropic in another form,” a source acquainted with the White House’s pondering told The Post on Tuesday.
Amodei, left, speaks with Salesforce CEO Marc Benioff during the keynote tackle at Salesforce’s Dreamforce convention at the Moscone Center on Tuesday. Getty Images
Multimillion-dollar fundraising
Effective Altruism’s obsessed adherents imagine in funneling their wealth toward the objectives they view as most important to humanity, including climate change and strict AI regulation.
Since its founding, METR and its associates have obtained funding from EA kingpins like Democratic megadonor Dustin Moskovitz and Skype cofounder Jaan Tallinn — both of whom are Anthropic traders.
In August, METR announced it had raised $71 million in new funding from exterior traders over the earlier six months.
Tallinn’s Survival and Flourishing Fund, a philanthropy group, has earmarked up to $752,000 in funding to help METR since its founding, including $324,000 in common money and a $428,000 matching pledge, according to public information.
Elsewhere, Moskovitz’s Coefficient Giving – previously recognized as Open Philanthropy before a hasty rebranding last yr – gave $1,515,000 to METR’s incubator, the ARC, in 2022. METR was initially recognized as ARC Evals before spinning off as an unbiased nonprofit and altering to its current identify in 2023.
METR staffer Pip Arnott is pictured. METR
METR doesn’t take any compensation from AI labs for its work, nor does it take donations from executives or workers of AI firms, the METR official said. In the case of Moskovitz’s donation to ARC in 2022, the official said the funding was firewalled and not used for METR’s operations.
That’s hardly reassuruing to critics.
“Saying you want to be audited by the ‘independent’ people at METR is a complete joke,” said Perry Metzger, chairman of Alliance for the Future, a Washington, DC-based AI coverage group. “It’s people who are funded by your buddies. It’s fully ridiculous.
“If you went out tomorrow and you’re a big bank and you had your audits done by an organization consisting entirely of your own former employees, which was funded by the same investors that are investing in you, do you know what the feds would do with you?” he added. “It would be positively medieval.”
Amodei’s shock proposal revealed last Saturday morning got here just days after Jacob Coxon, an Anthropic researcher who beforehand labored at OpenAI, abruptly give up the firm while warning that both AI giants have been “gambling with our lives” by pursuing progress without correct oversight.
Without safeguards in place, Amodei warned the web may very well be overtaken by AI bots within six to 12 months, “potentially causing hundreds of billions of dollars in damage.”
‘Stop pretending’
The doomsday warnings obtained sharp pushback in some corners of the tech and political world, with President Trump asserting that they have been a “hoax” and smaller AI startups arguing that leaders like Anthropic and OpenAI have been just attempting to rewrite the law to benefit themselves.
Hjalmar Wijk is chief scientist at METR. He presently works at “trying to reduce worst-case risks from future models” of different AI fashions, according to his LinkedIn profile. METR
Among the skeptics was former White House AI czar David Sacks, who called on Amodei to “stop pretending METR is independent when it is interviewed with Anthropic’s investors and staff.”
“Most of all, stop pretending the motivation to slow down is purely altruistic,” he added. “You face massive product-liability exposure if your products enable a truly damaging cyberattack.”
It’s also unclear if the AI industry’s push to police itself will go muster with US regulators.
Federal Trade Commission Chairman Andrew Ferguson warned that “everyone should be deeply suspicious” of any request to grant AI firms an antitrust exemption to coordinate security efforts.
“I will say generally, though, that if companies are simultaneously coming to Washington and asking for a host of regulations and an antitrust exemption, all of my alarm bells go off,” Ferguson said during a Tuesday event at Georgetown University.
Jasmine Dhaliwal is a coverage staffer at METR. METR
Under Amodei’s proposals, the third-party evaluators – which some have likened to nuclear inspectors – would “verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes,” according to his essay.
Megan Kinniment Kinniment, a member of METR’s technical employees, beforehand held a collection of positions at the Center on Long-Term Risk in London, where she centered on how “malevolent actors could increase the likelihood of poor AI outcomes,” according to her LinkedIn. METR
“Anthropic is unilaterally committing to this step now,” Amodei wrote. “We intend this to be part of a broader push to redouble efforts on our safety and alignment work.”
OpenAI’s Sam Altman said he agreed with Amodei about the “need to pace the frontier” – including that “committing to having independent evaluators with employee-like access is a great idea, and we will do the same.” He made no point out of METR.
Anthropic declined to remark.
Additional reporting by Jared Downing and Georgia Worrell
Stay informed with the latest in tech! Our web site is your trusted source for breakthroughs in artificial intelligence, gadget launches, software program updates, cybersecurity, and digital innovation.
For contemporary insights, professional coverage, and trending tech updates, go to us repeatedly by clicking right here.
