An alternative to the Turing Test, the Winograd Schema Challenge aims to determine how well an AI system handles common sense reasoning. The challenge tasks computer programs with answering a specific type of simple, common sense question called a pronoun disambiguation problem.
After a chatbot pretending to be a 13-year-old named Eugene Goostman “passed” a Turing test a few years ago, experts in artificial intelligence got together and decided that a traditional Turing test might not be all that effective in measuring the intelligence of a computer program after all. Instead, they came up with (among many other things) the Winograd Schema Challenge, which is intended to determine how well an artificial intelligence system handles commonsense reasoning: understanding the basics about how the world works, and implementing that knowledge in useful and accurate ways.
A few weeks ago, the very first Winograd Schema Challenge took place at the International Joint Conference on Artificial Intelligence in New York City. We spoke with Charlie Ortiz, director of the Laboratory for AI and Natural Language Processing at Nuance Communications and one of the organizers of the Winograd Schema Challenge, about how things went, why the challenge is important, and what it means for the future of AI.