OpenAI Will Form Special Team To Ensure ‘Superintelligent’ AI Doesn’t Go ‘Rogue’

Given the ever-growing popularity of artificial intelligence, it’s only natural that some are skeptical about the good the technology can bring. In fact, the biggest names in the industry—including Elon Musk and Steve Wozniak—have backed an open letter calling for a pause on training powerful algorithms.

Now, OpenAI, the creator of ChatGPT, one of the most popular AI chatbots, has announced it will be forming a new team with its chief scientist Ilya Sutskever in order to develop better safeguards and controls for “superintelligent” systems.

Incredibly or alarmingly—depending on how one views it—Stuskever and alignment team lead Jan Leike recently revealed in a blog post that they expect AI intelligence to exceed that of humans within the decade.

And because no one can say for sure if the technology remains 100% good (as many a movie has foretold), it’s necessary for researchers to begin looking into ways to better restrict machine-learning models from heading to the “dark side.”

“The vast power of superintelligence could… lead to the disempowerment of humanity or even human extinction. Currently, we don’t have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue,” they warned.

“Our current techniques for aligning AI, such as reinforcement learning from human feedback, rely on humans’ ability to supervise AI. But humans won’t be able to reliably supervise AI systems much smarter than us,” they explained.

As such, the company has decided to set up a Superalignment team, which will gain access to 20% of the computing the firm has to date. Overall, the team will work to solve challenges on how to control the rise of superintelligent AI better.

One method the group will be exploring is what is known as a “human-level automated alignment researcher,” whose main goal will be to train systems using human feedback before allowing the AI to evaluate other AI systems and ensure they don’t go off into the deep end.

“As we make progress on this, our AI systems can take over more and more of our alignment work and ultimately conceive, implement, study, and develop better alignment techniques than we have now,” Leike said in a previous blog post.

“They will work together with humans to ensure that their own successors are more aligned with humans. Human researchers will focus more and more of their effort on reviewing alignment research done by AI systems instead of generating this research by themselves,” they concluded.

Will this help quell fears of superintelligent AI eventually turning into a foe of humanity? Fingers crossed.

[via TechCrunch and Mint, cover image via Guble | Dreamstime.com]