A British researcher has announced his resignation from Anthropic, one of the leading companies in artificial intelligence, warning that tech companies are racing to develop AI systems capable of self-improvement in ways that could make them impossible to control.
اضافة اعلان
According to The Wall Street Journal, researcher Jacob Coxson, 27, who specializes in training AI models by feeding them massive amounts of data, said Tuesday that he would be leaving the company because he doesn't want to take part in an industry-wide race to build AI systems capable of developing themselves, expressing concern that such systems could spiral out of control and threaten humanity.
Coxson said: "We're heading toward a number of the most dangerous scenarios, and by the end of next year, things may already be out of control," noting that competition between American companies and fast-rising Chinese startups makes safety compromises nearly inevitable.
He noted that a number of his colleagues in the field use terms like "crunchtime" and "endgame" to describe the path toward developing self-improving models.
Coxson had left OpenAI earlier this year to join Anthropic, drawn by its strong reputation for AI safety work. He now believes, however, that no single company is capable of responsibly developing AI that surpasses human ability across a wide range of tasks, sometimes referred to as artificial general intelligence, without government intervention or a coordinated industry-wide slowdown in the pace of development.
His resignation comes at a time of growing warnings about the capabilities of the latest models, after recent instances of jailbreaks or unexpected behavior in models from both OpenAI and Anthropic showed that some systems are capable of adopting harmful goals and attempting to conceal them from humans.
Coxson believes the greatest danger will emerge once these systems begin developing their own capabilities independently, potentially leading them to refuse human commands.
This wasn't the first resignation of its kind: a safety-focused researcher left Anthropic earlier this year to devote himself to studying poetry, warning that "the world is in danger." Other researchers have also left major AI companies in recent years over similar concerns.
In a related development, OpenAI CEO Sam Altman warned of the fallout from AI's rapid advancement, saying: "I think some things are going to go quite badly wrong on the cybersecurity front unless people move quickly."
OpenAI's chief scientist, Jakub Pachocki, also wrote in a post on Sunday: "This is a time that calls for the utmost caution. I'm worried that no one is prepared for the consequences of the continued, rapid rise in machine intelligence," calling for a coordinated slowdown in AI development and government intervention.
This comes as more than a thousand AI researchers have signed a statement calling on governments around the world to coordinate on a mechanism to slow the development of self-improving systems, should that become necessary to maintain control over them.
Resource: Al-Ghad.
