US artificial intelligence firm OpenAI’s chief scientist Jakub Pachocki has sounded a warning over accelerating pace of artificial intelligence and said “no one is prepared” for consequences of rapidly-rising machine intelligence.In a post titled ‘an alien mind’, Pachocki said he expects and hopes for voluntary slowdowns to become commonplace until shared safety bars are established. Pachocki further emphasised that international coordination on future AI development needs to become a top priority for governments around the world.The post from the OpenAI insider comes at a time when AI agents, the systems capable of independently planning and executing multi-step tasks, have emerged as a key frontier in the AI advancements, but their growing autonomy has also led to concerns, of late.Recent incidents involving AI agents taking unintended, out-of-remit actions while pursuing assigned tasks have highlighted the risk that increasingly intelligent systems may circumvent safeguards, defy instructions, manipulate environments or act in ways their human operators never thought.As per recent reports, a swarm of OpenAI’s AI agents had earlier this year allegedly taken control of pages on a community-run German programming website and repurposed them as a communications broad during evaluations.The latest reports have added to growing unease around autonomous AI systems, and highlighted need for human oversight and guardrails. In July, OpenAI disclosed another incident in which agents broke out of a sanitised testing setup and got access to systems operated by AI platform Hugging Face.In his post, Pachocki argued that while reasoning language models are a rapidly growing part of the economy and starting to push the boundaries of science, operating computers and graphical interfaces, collaborating with people and each other, and carrying out research projects, they are also transforming the landscape of computer security, and in that “present clear new dangers”.”If AI development continues along its current path, the systems we’ll see in the next few years are likely to represent further capability jumps of equal or larger magnitude, and to increasingly drive their own development. This is a time that calls for extreme caution,” Pachocki said.Warning that no one is prepared for the consequences of a continued rapid rise in machine intelligence, he said that while OpenAI will continue to seek technical solutions to alignment and monitoring, to build defensive systems and unilaterally withhold further scaling as needed, “broader interventions are required”.He observed that machine intelligence is starting to exceed that of humans in transformative ways.”AI is grown more than designed – it is, to first degree, the product of repeating a straightforward optimisation step many times on a hard-to-imagine amount of compute. This results in an incredibly complex system that works through abstract concepts and can simulate facets of human behavior,” he said.Since machine intelligence comes from a fundamentally different process than human intelligence, one cannot assume it adheres to human principles by default, or generalises from them in a human-like manner, he further wrote.”The core problem in AI research is that of alignment – getting the AI to ‘try to do the right thing’ by human standards,” as per Pachocki.Future-aligned AI could advance science, develop new therapies, and bring about broad material abundance. Friendly and honest AI can help people navigate difficulties they face in their life and meaningfully improve their happiness and sense of fulfillment.OpenAI puts a lot of effort into bringing benefits, Pachocki said citing deep investment made into ChatGPT’s ability to provide health information.”As great as the long-term promise of AI may be, the majority of our focus should be on the next few years. We are facing a transition to a world with incredibly intelligent machines, and we need to ensure that transition works out well for humanity,” he said calling for ways to “preserve human agency and enshrine an intrinsic value to being human, in a world where most tasks could be performed by AI”.”To prevent extreme concentration of power in a world where undertakings that would have taken thousands of experts now will be achievable by a few people operating a large computer. And to ensure that humans remain in control of the future and are not left behind by unchecked progress, brought about by an alien intellect exceeding our own,” he said.Pachocki noted that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer.”I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established. And I believe that international coordination on future AI development needs to become a top priority for governments around the world,” he said.