The AI company Anthropic has been making major headlines recently. Its trillion-dollar IPO plan and its blood feud with secretary of defense Pete Hegseth have attracted much attention, but two other events may be even more consequential.
In early June, the company posted an article describing early signs of recursive self-improvement (RSI), a process in which an AI system devises ways to increase its own intelligence, leading to a greater ability to improve itself, and so on.
Obviously, uncontrolled RSI could produce a runaway feedback loop that leads to an irreversible loss of human control. Anthropic suggested the world should “slow or temporarily pause frontier AI development”. Then on 12 June, the White House issued an export control directive banning access to Anthropic’s new frontier models, Fable 5 and Mythos 5, for all foreign nationals – including many of its own key researchers. Anthropic responded by shutting the models down altogether.
These two June events are closely related. A few months ago, Anthropic’s Claude Code became good enough that its leading researchers no longer write any code at all; they just describe ideas and experiments to Claude and it does all the work.
This sped up the cycle of improvement – including the improvement of Claude Code itself – to the point where the latest iteration, called Mythos 5, showed the ability to conduct end-to-end cyberattacks with no human assistance. If such systems were released without cast-iron guardrails, almost anyone in the world could attack any country’s critical infrastructure at will.
These developments are only to be expected. They are symptoms of the inexorable increase in AI risk arising from the inexorable increase in AI capabilities. Yet, with the honorable exception of the UK’s AI Safety Summit in 2023, the world has largely been ignoring the risks.
The CEOs are telling us: “We’re on track to create superhuman intelligence, which has a good chance of causing human extinction.” (By “good chance” here, they mean a chance similar to the one in six chance of dying while playing Russian roulette with a loaded revolver; in this game, however, the revolver is pointed at all of our heads.) Yet governments reply: “That’s wonderful! Can we offer you a subsidy? Fast-track your permits?”
But finally, with the prospect of weapons of mass cyberdestruction in the hands of billions, the White House has reversed its deregulatory stance and suffered a rare attack of common sense.
They sputter: “Why did no one warn us about these AI systems?” Their response has been spasmodic, with an on-again, off-again executive order and now a ban on a system that had already been deployed, but the direction of travel is clear.
Unrestrained development of unsafe systems leads to intolerable risks. Governments can respond now, before the risks materialize, or they can wait and clean up the mess (if they still exist, that is). One leading AI CEO told me he didn’t expect…
Source link
Disclaimer
We strive to uphold the highest ethical standards in all of our reporting and coverage. We blogs.grocliq.com want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It’s possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.
Website Upgradation is going on for any glitch kindly connect at [email protected]
