Who Is Watson? Standing Up to the Machine, One Trivial "Jeopardy!" Clue at a Time

I did it for all of us.

Lately you may have been hearing all the buzz about Jeopardy! -- a rare phenomenon. The show is one of those quiet American institutions that stay out of the headlines, plodding along with benign, unglamorous steadiness. It’s like minor league baseball, or Law & Order, or Punxsutawney Phil – even if you haven’t gone out of your way to check it out in a while, you know it’s doing its thing pretty much as you left it, and that’s comforting.

The last time the show became news was in 2004, after the show changed its formula, lifting its traditional five-game limit. In June, one Ken Jennings came along and won five games… and then kept right on going, through the show’s summer break, and into the fall. Suddenly Jeopardy! was an object of media frenzy and public conversation. When word leaked that he’d finally lost, the airing of Ken’s Last Game (#75) at the end of November 2004 became a real event.

Many were pleased to see him fall. He got overconfident, bet too big on a few daily doubles, missed a makeable Final Jeopardy… and the predominant emotion was relief that he was not going to be the permanent champion. He was only human after all.

And the next night, it was the same old Jeopardy! again. Comfortable, comforting.

Well, the Jeopardy! folks have now messed with the formula in an even more fundamental way. In 2004, Ken Jennings played like a machine.  However, for three games next week, one of the contestants will be an actual machine.

The machine – something like 30 computers linked to make one supercomputer – is named Watson, as a tribute to IBM’s founder Thomas Watson.  IBM and Jeopardy! have teamed up to hype the showdown. It’s not quite the Super Bowl, but it sure seems to be getting into the public consciousness.

In the 1990s IBM created Deep Blue, the chess-playing supercomputer. It lost to world champion Garry Kasparov, but then came back and beat him. Score one for the machines. It was impressive, but the game of chess is limited. Tough as it is, it’s computer-friendly -- there are only so many moves.

Jeopardy! is a different story. Any topic at all can be the subject of its questions (or put more accurately, its answers). People often ask if contestants are told the categories ahead of a particular game. Um, no. That’s part of the challenge.

By the year 2011, we’re all familiar with internet search engines, and even variants that can answer simple questions. If you type ‘what is the capital of Macedonia?’ into Google, the first few results will make it clear that it’s Skopje you’re looking for. (Well, unless you’re talking about the Greek province of Macedonia instead – which I'd classify as human error).

But Jeopardy! is much more subtle: the questions are sneaky, relying on puns, drawing on strange connections and hidden linkages. That’s why IBM decided to use it as a way to test its new artificial intelligence effort, known as Deep QA (here’s an interesting video explaining it).

Watson’s big test will come in three games against Ken Jennings and another big winner, Brad Rutter. Tune in February 14-16 to see the main event. (And for some of us alpha nerds, yes, that first show IS the perfect way to celebrate Valentine’s Day.)

By the way, the winner gets a million bucks, and the payouts for second and third place aren’t bad either. Winningly enough, IBM is donating all of Watson’s earnings to two remarkable charities: faith-based international relief and development organization World Vision and an organization that allows people to donate the unused processing power when their computers are idle to somehow create clean energy (!). Even more impressively, Jennings and Rutter have said they’ll donate half of what they win to charity.

To warm him (if you don’t object to calling it ‘him,’ since he has a name and all) up, IBM and Jeopardy! had Watson play 55 practice games against former champs. I was one of them. These will never be televised, but if you’re a trivia head and you have a chance to play Jeopardy!, you pretty much take it.

One morning in November, I walked into the IBM Research Campus in the beautiful Hudson Valley north of New York City. In the lobby, I saw IBM’s founding exhortation: “THINK.” It seemed like a good plan.

Three other ex-champions were there as well. I was embarrassed to admit that I didn’t remember seeing them on the show because, well, I haven’t watched it a whole lot since I was on it.

They had created a mini-Jeopardy! set for us. No Alex Trebek, though; they hired a comedian to play his role. Another change from the real thing was the small audience… all of whom were rooting for Watson. Because they had built him.

The other human and I flanked Watson. Watson was represented by an avatar, so he was 8 feet tall and blue, and spoke Na’vi.

Sorry, stale 2009 humor there. The avatar was a disk suspended from the ceiling, with colored lights coursing over its surface. The colors varied according to how confident he was in a particular answer; those in the audience could watch monitors which listed Watson’s top few possible answers, ranked with the probability they were correct. For each question, depending on how he was doing in the game, Watson would decide how confident he had to be in order to ring in. In a close game, he might insist on 90% confidence, while in a blowout he might try questions on which he was only 80% confident. That’s what we human players try to do too, but we’re much less precise and efficient.

As the first game got going, Watson showed he was very, very good. This was no surprise, as we’d been told he was winning 70% of the games, all against former multi-game winners. Even on the twistiest, punniest answers, Watson could ascertain what information was being sought and answer with few mistakes. He had a mechanical plunger that had been rigged to resemble the buzzers (Jeopardy! likes to call them ‘signaling devices’) people use. But he cleaned our clocks, winning most of the clues we all knew. We were assured it was a fair fight and that Watson was just ringing in faster because he was more confident. We were skeptical, but there was no way to test it.

Game 1 was a complete runaway, so Final Jeopardy couldn’t affect the outcome. The clue asked what performer the London Symphony Orchestra backed on his 2010 ‘Symphonicity’ tour. For most of us who are steeped in 1980s music, it’s easy:  the reference is to the ‘Synchronicity’ album by the Police, so the clue was looking for Sting. But Watson didn’t get it. Okay, not everyone is a big 80s fan.

In game 2, the other player and I made it a bit more of a match, but Watson continued to cruise. He was alarmingly good, hitting all the annoying Rhyme Time clues, such as one on Presidential posteriors for which he correctly chimed in “What is Bush tush?” While a bit closer than the first game, it was still not close enough for Final Jeopardy to matter.

Yet Watson once again blew it. The category was “The Hall of Presidents,” and the clue: “Of the 9 presidents whose images have a beard or mustache, this late 19th century man is the only Democrat.” If you know there was only one Democratic President of the U.S. in the late 19^th Century – the walrus-mustachioed Grover Cleveland—it’s a tough one to miss.

Watson’s answer: “Who is Fidel Castro?” He didn’t assume – as we humans did without a second thought – that the clue was about a U.S. President, and he apparently found so many references to Castro and beards in his databases that he couldn’t resist – even though Castro wasn’t even in the right century.  It was intriguing because it wasn’t a mistake any human player would make. But again, the game wasn’t close enough for Watson’s error to affect the outcome.

A former champion named Bill McDonald joined Watson and me for Game 3, and for the first time, it wasn’t a runaway. It helped that a seemingly ‘overconfident’ Watson bet big and missed on a couple of Daily Doubles. He was still in first place heading into Final Jeopardy, and in a fluke, Bill and I were tied for 2^nd place.  For the first time, the pressure was on Watson.

The category was 20^th Century Artists. The clue: “In 1950 he answered a Time magazine article on him, & a common criticism, with a telegram reading, ‘No chaos damn it’.” Immediately I identified Jackson Pollock as the most obviously ‘chaotic’ artist of the era, as did Bill; we both bet all we had, and turned out to be right.

Watson’s answer was revealed:  “Who is Albert Einstein?” Despite his dominance of the regular Jeopardy! clues, once again, the Final Jeopardy question had been just a bit too subtle for him.

Bill and I (as co-champions) had vanquished mighty Watson. Watson is quite a machine… but even though his databases stretch as far as the eye can see, he still can’t make the connections our brains make in an instant.

So count that as a tiny victory for the humans against the machines that play such a massive and growing role in our lives. Like I said, I did it for all of us.

You’re welcome.