Several days ago KitGuru reported on the new supercomputer, created by IBM to outsmart two of Jeopardy's finest human players. Was it a success?

Watson, the nickname for the IBM supercomputer butchered two of Jeopardy's greatest champsions, Ken Jennings and Brad Rutter. This test of new artificial intelligence has been a resounding success for IBM as it earned $35,734, blowing away Rutter and Jennings, who had earned $10,400 and $4,800 respectively.

It wasn't all a success, Watson failed to answer the last Jeopardy clue was was in the category of US Cities. The clue was “Its largest airport was named for a World War II hero; its second largest, for a World War II battle.”

The human players got it right by saying ‘What is Chicago?”, for O'Hare and Midway, but Watson said “What is Toronto???”, complete with three question marks.

Technically, IBM launched an investigation as to how the machine got it so wrong, and David Ferucci, the manager of the Watson project at IBM research said on the blog that several things probably confused the computer, as reported by Steve Hamm.

First, the category names on Jeopardy! are tricky. The answers often do not exactly fit the category. Watson, in his training phase,  learned that categories only weakly suggest the kind of answer that is expected, and, therefore, the machine downgrades their significance.  The way the language was parsed provided an advantage for the humans and a disadvantage for Watson, as well. “What US city” wasn’t in the question. If it had been, Watson would have given US cities much more weight as it searched for the answer. Adding to the confusion for Watson, there are cities named Toronto in the United States and the Toronto in Canada has an American League baseball team. It probably picked up those facts from the written material it has digested. Also, the machine didn’t find much evidence to connect either city’s airport to World War II. (Chicago was a very close second on Watson’s list of possible answers.)  So this is just one of those situations that’s a snap for a reasonably knowledgeable human but a true brain teaser for the machine.

KitGuru says: To be fair, this was the only real glitch in the show with Watson dominating the performance and the researchers were happy that Watson seemed to ‘guess intelligently' at the end of the game.

Become a Patron!