Presuming Watson

I’m not a huge Jeopardy fan. I’ve still watched the show many times over the years, however, originally as part of some never-actually-developed-in-real-life fantasy of applying to be a contestant and then eventually merely as a play-along-at-home type. Sometimes I win. Sometimes I don’t win. I have the sense it’s dramatically easier to play from the couch than it would be actually to compete for real on the actual program. Still, when I do watch I usually enjoy the show. And I’ve had plenty of time to form an opinion, as have had we all; the show was first shown on television in 1964 in the version featuring Art Fleming as the host and is currently in the twenty-seventh consecutive season of its current incarnation with host Alex Trebek.

I watch. I don’t watch. Jeopardy is not a big part of my life. Up until this week, I don’t think I could have even said with any certainty when exactly the show is on or on what channel. But I’ve been watching all week this week, as I know many of you also have, as the IBM supercomputer named Watson was one of the featured contestants. The other two contestants, both human beings, were no slouches. (One is the person who has won the most money on the game; the other is the one who has won the most games. Together they’ve gone home with more than five million dollars between them. For its part, the computer not only hasn’t ever won a penny but also won’t go home, so to speak, with anything this week either because IBM has pledged to give all of Watson’s winnings to charity.)

Watson is a big machine both physically and in terms of what it can do. It can perform eighty trillion operations per second. To put things more clearly, the machine has the equivalent of two million pages of information in its memory and can scan them all in about three seconds. (Computer savvy readers will be interested in knowing that Watson operates with fifteen terabytes of RAM. By comparison, the machine I am using to write this has four gigabytes of RAM, which means that Watson operates three-thousand-seven-hundred-and-fifty times faster than my computer. And this is actually a pretty zippy laptop I write on!) Watson is not linked to the internet, which would obviously have given it an unfair advantage on the show. Whatever answers it gave, it had therefore to find somewhere within the facts it already “knew,” just as the human contestants were obliged to do. So the contest was fair, assuming you don’t think one contestant having a brain the size of ten refrigerators lined up next to each other made the playing field sufficiently unlevel to make meaningless the results.

On the first night, the human beings and the machine appeared more or less evenly matched. At the end of the first round of play, in fact, the machine was actually tied with one of the human beings for first place. (Each had won $5000 and the third player had won a measly $2000. But these were only make-believe sums used to keep score; the real prize was $1,000,000.) By the second night, however, Watson roared dramatically into first place with a score of almost $36,000, while the human contestants ended up with less than half of that between the two of them. It was, I have to admit, a very exciting game. By the third game, played Wednesday evening, it was a rout and, in the end, the win went to the machine and the million went to charity. (Watson ended up with more than $77,000, while neither of the human competitors ended up with more than $25,000.)

What was of interest to me, though, was not so much that the computer answered a lot of questions correctly. After all, isn’t that what computers do, process information and produce it on demand? Okay, it’s incredibly impressive that the machine understands human speech, that it can deal with the nuance and wordplay that characterize a fair number of the clues on Jeopardy, and that it has the capacity to bet wisely when called upon to do so by analyzing the chances that it has the right answer and then placing its wager accordingly. But what was even more interesting to me were the things the computer messed up.

On the second evening of play, the machine made an astounding error. The category was “U.S. Cities.” The answer was “It has two airports, one named after a World War II hero and the other named after a World War II battle.” The correct answer, which both human beings knew, was Chicago. (O’Hare is named Lieutenant Commander Edward O’Hare, a World War II flying ace; Midway Airport was named in honor of the American servicemen and women who fought in the Battle of Midway, one of the most decisive naval encounters of the war.) But the computer answered, insanely, “What is Toronto?”, apparently unaware that Toronto is not a U.S. city at all. The IBM people had some instant answers of their own to explain the slip-up. There are, they noted, cities named Toronto in the U.S. (So what? None of them has two airports.) The Canadian Toronto, they further noted, has a baseball team that is part of the American League. (So what? Doesn’t the machine know that not all baseball teams in the American League are located in the U.S.? And what did the question have to do with baseball?) And it is also so that the real Toronto actually does have an airport named for a war hero, Billy Bishop, who was Canada’s most celebrated World War I pilot. (So what? The question referenced World War II, not World War I. And the other airport is named for Lester Pearson, a former Canadian Prime Minister, not a battle.) But they took comfort, which I do have to say is more than fully justified, in the fact that the machine was able to realize that it was giving a poor answer that was probably wrong and so wagered less than a thousand dollars, thus retaining its great lead even after having given a wrong answer.

The first night, Watson also gave a crazy response. The “answer” was “This word denotes both stylish elegance and students who graduate in the same year.” The correct question was obviously “What is class?” For what it’s worth, I got it right. The computer, however, came up with the meaningless “What is chic?” (The IBM people no doubt had an explanation for that as well, although I couldn’t find one published on the web anywhere.) On the other hand, in a practice round the machine was able to respond to the confusing clue “A Green Acres star goes existential (and French) as the author of The Fall” correctly. (The answer, obviously, is Eddie Albert Camus. The topic was “All Eddie-Before and After,” which barely means anything at all out of context. Now that is impressive. Maybe I would have gotten it. Maybe not. Okay, probably not. Of course, now that I know the answer, I will definitely get it right if anyone ever asks.)

What is worth noticing is that even with its unimaginably powerful memory, the machine didn’t know everything. It was incredibly good at answering questions. It was even good at figuring out puns and jokey plays on words to arrive at the right answer. But, in the end, it made errors that its human programmers had apparently failed to anticipate it would. Now, of course, they will fix things so that it will take those pesky categories more into account and avoid giving answers that correspond to only x or y in an “x and y” clue. But then more things will crop up. And then more things too after that.

After watching two nights of Jeopardy, I am awe-struck by IBM’s achievement of having created Watson, but I believe that what has been attained is far more the ability to teach a machine to mimic the human reasoning process incredibly well than the actual investiture in a machine of the actual human ability to reason. Does it come to the same thing? I am precisely the right age, as I know also are many of my readers, to have been extremely taken as a teenager with Stanley Kubrick’s 2001: A Space Odyssey when it came out in 1968 and to have assumed almost casually that HAL, the super computer in the “Jupiter Mission” part of the movie, was eventually going to exist once human beings learned how to create such a thing. HAL, you may recall, had the full measure of intelligence of the brightest human being but was also able to develop emotions such as love, hate, envy, and fear. And, as a result, it eventually also became susceptible to mental illness and, indeed, suffered a nervous breakdown that was terrifyingly depicted in the movie. So what I would really like to see is not a Jeopardy match featuring two really smart human beings and Watson, but a triple-machine show featuring HAL, Watson, and possibly Deep Blue, the supercomputer that defeated Gary Kasparov at chess in 1997. Now that would be a great contest. Deep Blue was a very smart machine. Watson, by all accounts is even smarter. But my own money would be on HAL (or rather would be on HAL if there really was such a thing in the world) even if advancing each letter in his—I mean, its—name forward by one spot in the alphabet does somehow turn HAL into IBM.

Clearly, 2001 has come and gone without HAL existing. Now it’s ten years later, and Watson still thinks that kids who graduate high school together are called a chic, not a class. Artificial intelligence clearly still has a long way to go before the word “artificial” in that phrase becomes meaningless.