Iconary: Collaborate and Communicate with AllenAI

Carissa Schoenick
Feb 5, 2019 · 3 min read

Oren Etzioni, Ani Kembhavi, Carissa Schoenick

Recent advancements in AI are being powerfully demonstrated in the realm of games — from classic board games like Chess and Go to complex multiplayer online games like DOTA 2 and StarCraft, sophisticated AI systems are pitted against human players, leveraging ever more complex algorithms to outmaneuver their human adversaries. These AI models are impressive, but limited. What if a game-playing AI system could combine several useful, real-world skills like language comprehension, visual understanding, and common sense? What if this system were collaborative, teaming up with human partners instead of being pitted against them? This is the motivation behind Iconary, a new project from AI2 (iconary.allenai.org).

Collaborator instead of antagonist — AllenAI draws and guesses with you.

AI2 just released a novel AI system this week we call AllenAI, designed to be able to both draw and guess along with human players in a Pictionary-style game called Iconary. Iconary offers players a limited set of icons along with a phrase describing a situation, like “crowd celebrating a victory.” Players need to use the icon set to compose a scene that represents the phrase, trying to encourage their partner to guess it correctly. AllenAI can play both roles in this game, generating unique scenes from this library of icons for the human player to guess, as well as guessing phrases based on scenes that the human player composes. When playing the role of the drawer, AllenAI can also update its compositions based on its human partner’s guesses to help successfully guide them towards the correct phrase.

AllenAI has never before encountered the unique phrases in Iconary, yet our preliminary games have shown that our AI system is able to both successfully depict and understand phrases with a human partner with an often surprising deftness and nuance. This feat requires combining natural language understanding, computer vision, and the use of common sense to reason about phrases and their depictions within the constraints of a small vocabulary of possible icons. Being successful at Iconary requires skills beyond basic pattern recognition, including multi-hop reasoning, abstraction, collaboration, and adaptation.

Iconary is outlining an exciting research platform for highly relevant issues in AI research which Go, Chess, and other online video games can’t touch — a collaborative experience situated in real-world knowledge. As the world meets AllenAI and plays Iconary, we will learn even more about the way the players communicate with one another, allowing us to continue to improve our models and to move Iconary toward a collaborative experience so natural and compelling you wouldn’t know there wasn’t a human on the other end. We plan to expand the phrase set and icon vocabulary of the game as we raise the bar by introducing more conceptually challenging terms like “roommate” and “governor.” We hope Iconary encourages new lines of research within both AI2 and the wider AI research community around this compelling intersection of vision, language, and reasoning.

Please Give Iconary a try at iconary.allenai.org, and stay tuned for the next evolution of this important milestone in AI from AI2.

Learn more about Iconary from AI2

AI2 Blog

AI for the common good.

Carissa Schoenick

Written by

Sr. Program Manager & Communications Director at AI2

AI2 Blog

AI2 Blog

AI for the common good.

More From Medium

More from AI2 Blog

More from AI2 Blog

More from AI2 Blog

Green AI

Feb 14 · 9 min read

8

More from AI2 Blog

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade