MIT Technology Review/Will Knight: How can we be sure AI will behave? Perhaps by watching it argue with itself.

MIT Technology Review   

A new website lets anyone play the role of an argumentative AI.

OpenAI blog
Intelligent Machines
How can we be sure AI will behave? Perhaps by watching it argue with itself.
Experts suggest that having AI systems try to outwit one another could help a person judge their intentions.

    by Will Knight May 3, 2018

Someday, it might be perfectly normal to watch an AI system fight with itself.

The concept comes from researchers at OpenAI, a nonprofit founded by several Silicon Valley luminaries, including Y Combinator partner Sam Altman, LinkedIn chair Reid Hoffman, Facebook board member and Palantir founder Peter Thiel, and Tesla and SpaceX head Elon Musk.

The OpenAI researchers have previously shown that AI systems that train themselves can sometimes develop unexpected and unwanted habits. For example, in a computer game, an agent may figure out how to “glitch” its way to a higher score. In some cases it may be possible for a person to supervise the training process. But if the AI program is doing something impossibly complex, this might not be feasible. So the researchers suggest having two systems discuss a particular objective instead.

“We believe that this or a similar approach could eventually help us train AI systems to perform far more cognitively advanced tasks than humans are capable of, while remaining in line with human preferences,” the researchers write in a blog post outlining the concept.

Take, for instance, an AI system designed to defend against human or AI hackers. To prevent the system from doing anything harmful or unethical, it may be necessary to challenge it to explain the logic for a particular action. That logic might be too complex for a person to comprehend, so the researchers suggest having another AI debate the wisdom of the action with the first system, using natural language, while the person observes. Further details appear in a research paper.

Having AI programs argue with one another requires more sophisticated technology than exists currently. So thus far, the OpenAI researchers have only explored the idea with a couple of extremely simple examples. One involves two AI systems trying to convince an observer about a hidden character by slowly revealing individual pixels.

The researchers have created a website where any two people can try playing the roles of the debating AI systems while a third serves as the judge. The two participants compete to convince the judge about the nature of an image while highlighting parts of it. Eventually it becomes easier for the observer to tell who is being honest.

Vincent Conitzer, a researcher at Duke University who studies ethical issues involving AI, says the work is at an early stage but holds promise. “Creating AI systems that can explain their decisions is a challenging research agenda,” he says. “If successful, it can greatly contribute to the responsible use of AI.”

As it stands—and despite some outlandish statements from the likes of Elon Musk (an OpenAI funder and until recently a member of its board)—we are still a long way from having AI systems capable of deceiving and outwitting us in the type of scenario portrayed in movies like Ex Machina and Her.

Still, some AI researchers are exploring ways of ensuring that the technology does not behave in unintended ways. This may become more important as AI programs become more complex and inscrutable (see “The dark secret at the heart of AI”).

“I think the idea of value alignment through debate is very interesting and potentially useful,” says Ariel Procaccia, a professor of computer science at CMU who studies decision making with autonomous systems.

However, Procaccia notes that the work is very preliminary, and that the concept may even contain a fundamental contradiction. “In order to debate value-laden questions in a way that is understandable to a human judge, the AI agents may need to have a solid grasp of human values in the first place,” he says. “So the approach is arguably putting the cart before the horse.”

Iyad Rawan, a researcher at MIT’s Media lab, adds that the researchers would need to be careful that a pair of AIs didn’t get into a circular argument. “I do think they’ll hit some very tricky issues very quickly,” he says. “First in how to automate argumentation in natural language, which is still an unsolved problem.”

Meet the Experts in AI, Robotics and the Economy at EmTech Next.
Learn more and register
Share

Will Knight

Will Knight Senior Editor, AI

Will Knight is MIT Technology Review’s Senior Editor for Artificial Intelligence. He covers the latest advances in AI and related fields, including machine learning, automated driving, and robotics. Will joined MIT Technology Review in… More
Related Video

More videos

Intelligent Machines
Blood-Delivering Drones 04:23

Intelligent Machines
AI’s Economic Impact 30:23

Intelligent Machines
Roundtable: Expanding the Reach of AI’s Benefits 20:38
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

    Facebook helped create an AI scavenger hunt that could lead to the first useful home robots

    To make AI programs smarter, researchers are creating virtual worlds for them to explore.

    by Will Knight
    One way to get self-driving cars on the road faster: let insurers control them

    Data gathered by autonomous cars and shared with insurance companies could be used to keep the vehicles from taking undue risks.

    by Jamie Condliffe
    How a gaming chip could someday save your life

    Nvidia has made a fortune selling chips for games and AI. Now it wants to reboot hospitals.

    by Will Knight

More from Intelligent Machines
From Our Advertisers

    Provided by BBVA
    Innovations, Ideas, and Insights

    In partnership with Qualcomm
    On-Device Processing and AI Go Hand-in-Hand

    Presented in partnership with Qualcomm
    Making On-device AI Ubiquitous

Want more award-winning journalism? Subscribe and become an Insider.

    Insider Plus {! insider.prices.plus !}* Best Value

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus the digital magazine, extensive archive, ad-free web experience, and discounts to partner offerings and MIT Technology Review events.
    {! insider.buttons.plus.buttonText !}

       
    See details+
    Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning print magazine, unlimited online access plus The Download with the top tech stories delivered daily to your inbox.
    {! insider.buttons.basic.buttonText !}

       
    See details+
    Insider Online Only {! insider.prices.online !}*

    {! insider.display.menuOptionsLabel !}

    Unlimited online access including articles and video, plus The Download with the top tech stories delivered daily to your inbox.
    {! insider.buttons.online.buttonText !}

       
    See details+

* {! insider.display.footerLabel !}

See international prices
The Algorithm News and views on the latest in artificial intelligence

Follow us
Twitter Facebook RSS
MIT Technology Review

The mission of MIT Technology Review is to equip its audiences with the intelligence to understand a world shaped by technology.

Browse
International
Editions

    Company
    Your Account
    Customer Support
    More
    Policies

MIT Technology Review © 2018 v.|eiπ|
/3
You've read of three free articles this month. Subscribe now for unlimited online access. You've read of three free articles this month. Subscribe now for unlimited online access. This is your last free article this month. Subscribe now for unlimited online access. You've read all your free articles this month. Subscribe now for unlimited online access. You've read of three free articles this month. Log in for more, or subscribe now for unlimited online access. Log in for two more free articles, or subscribe now    for unlimited online access.