WATSON on Jeopardy! In February!

Watson being a computer.

http://www.kurzweilai.net/ibm’s-watson-will-compete-on-jeopardy-in-february

IBM’s ‘Watson’ will compete on Jeopardy! in February
December 14, 2010 by Editor
IBM and America’s Favorite Quiz show Jeopardy! today announced that an IBM computing system named “Watson” will compete on Jeopardy! against the show’s two most successful and celebrated contestants — Ken Jennings and Brad Rutter.

The first-ever man vs. machine Jeopardy! competition will air on February 14, 15 and 16, 2011, with two matches being played over three consecutive days.

Watson, named after IBM founder Thomas J. Watson, was built by a team of IBM scientists who set out to accomplish a grand challenge – build a computing system that rivals a human’s ability to answer questions posed in natural language with speed, accuracy and confidence. The Jeopardy! format provides the ultimate challenge because the game’s clues involve analyzing subtle meaning, irony, riddles, and other complexities in which humans excel and computers traditionally do not.

Watson is a breakthrough human achievement in the scientific field of Question and Answering, also known as “QA.” The Watson software is powered by an IBM POWER7 server optimized to handle the massive number of tasks that Watson must perform at rapid speeds to analyze complex language and deliver correct responses to Jeopardy! clues. The system incorporates a number of proprietary technologies for the specialized demands of processing an enormous number of concurrent tasks and data while analyzing information in real time.

Competing against Watson will be two of the most celebrated players ever to appear onJeopardy! Ken Jennings broke the Jeopardy! record for the most consecutive games played by winning 74 games in a row during the 2004-2005 season, resulting in winnings of more than $2.5 million. Brad Rutter won the highest cumulative amount ever by a single Jeopardy! player, earning $3,255,102. The total amount is a combination of Rutter’s original appearance in 2002, plus three Tournament wins: the “Tournament of Champions” and the “Million Dollar Masters Tournament” in 2002 and the “Ultimate Tournament of Champions” in 2005.

The grand prize for this competition will be $1 million with second place earning $300,000 and third place $200,000. Rutter and Jennings will donate 50 percent of their winnings to charity and IBM will donate 100 percent of its winnings to charity.

“After four years, our scientific team believes that Watson is ready for this challenge based on its ability to rapidly comprehend what the Jeopardy! clue is asking, analyze the information it has access to, come up with precise answers, and develop an accurate confidence in its response,” said Dr. David Ferrucci, the scientist leading the IBM Research team that has created Watson. “Beyond our excitement for the match itself, our team is very motivated by the possibilities that Watson’s breakthrough computing capabilities hold for building a smarter planet and helping people in their business tasks and personal lives.”

“We’re thrilled that Jeopardy! is considered a benchmark of ultimate knowledge,” said Harry Friedman, Executive Producer of Jeopardy!. “Performing well on Jeopardy! requires a combination of skills, and it will be fascinating to see whether a computer can compete against arguably the two best Jeopardy! players ever.”

Jeopardy!, the winner of 28 Emmy awards since its syndicated debut in 1984, is in the Guinness Book of World Records for the most awards won by a TV Game Show. The series is the #1-rated quiz show in syndication with nearly 9 million daily viewers. Jeopardy! is produced by Sony Pictures Television, a Sony Pictures Entertainment Company. It is distributed domestically by CBS Television Distribution and internationally by CBS Television International, both units of CBS Corp.

Beyond Jeopardy!, the technology behind Watson can be adapted to solve problems and drive progress in various fields. The computer has the ability to sift through vast amounts of data and return precise answers, ranking its confidence in its answers. The technology could be applied in areas such as healthcare, to help accurately diagnose patients, to improve online self-service help desks, to provide tourists and citizens with specific information regarding cities, prompt customer support via phone, and much more.

Prepping a Machine to Play a Human

This fall, Watson played more than 50 “sparring games” against former Jeopardy! Tournament of Champions contestants in final preparation for its television debut. In addition, Watson has taken and passed the same Jeopardy! contestant test that humans take to qualify to play on the show, giving Jeopardy! producers confidence that the match will be both entertaining and competitive. Highlights of the sparring matches can be viewed and tracked over the next few weeks at www.ibmwatson.com.

https://www.youtube.com/watch?v=FC3IryWr4c8

Very excited!

this rules

DUMPLINGS!

I mean to post that on another thread. Sorry.

Woah...wondering what's inside the box. Presumably the speech-to-text component is good, and they have a dump of Wikipedia/the web/whatever, but I'm very surprised the QA system is competitive with good human contestants, assuming the questions are not completely formulaic (not too familiar with Jeopardy). There are some basic linguistic phenomena that natural language understanding systems just cannot handle.

Man, what does IBM even do these days?

Yeah, seems like they'd be better off having it just play regular people on Jeopardy for a while and forego the whole Ken Jennings thing! Seeing it even sort of manage against vaguely competent players would be impressive enough.

and based on that video they have a ways to go

I am wondering if it'll have problems buzzing in at the proper time. Isn't that something that a lot of former contestants talk about, how there's kind of a peculiar timing that's required?

Yes.

Yeah if a human can buzz in before the whole answer's read out, how is a computer gonna manage that? I mean maybe thats the point here, that its impressive enough to be predictive and whatnot.

That's gotta be the least of their problems!

I figure it might do okay with things like tonight's "A + 4 letters" category - if it can understand the parameters of the category, then it shouldn't be too hard to keep up with the associative/keywordiness of the clues and just skim through the dictionary. And some of them are narrow enough trivia that I could imagine it working - like, y'know, "In winter of such and such year in such and such city, the first one of THESE was held" - - basically, if you had the speed of a computer to Google the year, the city, and Winter, you'd be really close to the answer.

xpost

Yeah if a human can buzz in before the whole answer's read out

No, the way it works is that you have to wait until Alex finishes reading the clue, and if you buzz in too early, you get locked out for like a quarter of a second. (On the first Trebek season, in 1984-85, you could buzz in whenever, but it didn't make for particularly good TV.) So presumably this won't be an issue -- Watson will be working with the same complete set of information as everyone else.

I guess I am wondering about the timing, though. If Watson knows the right answer, will it always be able to buzz in at exactly the appropriate moment? (Also, does it ever wait and "think" about it before buzzing in?)

Ah right I didnt know that - yeah that helps. The YT clip was great. This is so cool.

Judgment Day next week! ZOMG!!! HUMANS PREPARE TO DIE!!!!!!!!!!!

personally i can't wait because i'm going to edit the watson episodes so that watson keeps giving really obnoxious answers and alex gets pissed while ken smugly presses on

also, watson's voice will be 8x louder than either alex, ken or the other contestant's voices

lol z_s wins

kind of in this style:
https://www.youtube.com/watch?v=O9f1zM7I8Z8

my goal is to get a video up in the middle of next week and hopefully get a million views for people searching for "Jeopardy Watson Computer"

watson is killin all yall jive turkeys

Watson!!!

how did our boy do

seems kinda unfair that he doesnt get a text file of an opponents answer if they get it wrong

also this sorta felt like a long commercial for ibm

seems kinda unfair that he doesnt get a text file of an opponents answer if they get it wrong

― johnny crunch, Monday, February 14, 2011 8:49 PM (3 minutes ago)

haha yeah alex is like "uh no...ken already guessed that"

yeah but it's not like he "hears" that either!

what is the score?

watson 5000
brad 5000
ken ~2000

man ken really doesn't have it anymore

did it air?

yeah it was on today, i guess it's just like a practice round or something tho?

its gonna be 2 complete games spread over 3 days of shows iirc

ken threw the practice round so he could bank in las vegas

best part was watson found the daily double on his first category pick on a full board i mean c'mon man

yeah that was ridic

watson went hard in the paint

big up

omg @ Z S upthread -- fantastic

this was waaay more fun to watch having actually been on jeopardy recently

i'm still unclear as to how watson knows when to ring in: presumably they send it some kind of signal, even though it can't see the lights? getting used to trebek's cadence was definitely the hardest part of it for me; as i'm sure jaymc can attest, all the onstage practice is hosted by a dude who talks nothing at all like trebek

One thing that is clear is that Brad is really ridiculously good and would probably have won a bunch of games in row (okay maybe not 75+) if he'd be allowed to back in his day.

Was kind of feeling bad for skintubes in the beginning, but it seemed like they were getting used to the way Watson rang in.

i'm still unclear as to how watson knows when to ring in

Yeah. Maybe some voice recognition software? There has to be some analog element to it since Brad sometimes was buzzing in before W even if W's confidence level met its threshold.

I'm betting that Watson out-buzzed both of them quite a few times.

Yeah, and the other guys got in before Watson, too.

Well, I mean, the humans can only buzz in at a certain official time too - there's a light that switches on or something, right? Before which time, if you buzz in, it locks you out for the first second or two of the actual "open buzz-in" time. So it's not just like, "when Alex is done reading you can buzz in" - there's a hard, 0 or 1 style limit that would be easily converted to a digital thing.

Somebody must be assembling all the times Watson was wrong and figuring out what they have in common, yes? It doesn't seem like obscurity of information was the issue, so it has to be a wording/complexity/subtlety of meaning issue?

I was mildly annoyed that "Maxwell's Silver Hammer" was accepted for what, syntactically, should have been Maxwell," but Alex accepts that stuff all the time from human contestants, so hey.

The only bummer with seeing the confidence level score is that the player at home (aka me) has an even shorter window to come up with the answer.

hahaha that was bothering me so much! /dork

when i saw this thread on new answers the first few times i was like "there's someone on ilx named Watson? what display name do they post under?"

then when i turned on Monday's show and saw the graphics with the three answers and the percentages next to them at the bottom of the screen, i went "woah, what's going on, is this a Who Wants To Be A Millionaire-style special edition of Jeopardy?". it took me a good 30 seconds to put 2 and 2 together.

lol that website spelled her name wrong even though she literally wrote it out in that picture

No bump for yesterday's episode? Ken actually jumped out to a big lead in the first round; the machine seemed pretty stumped for a bunch of questions. Of course Watson picked it up in double Jeopardy and then nailed FJ; still, it looked to me like Ken knew the answer to almost every question but couldn't buzz in on time. "I for one welcome our new computer overlords" - priceless

yeah i actually got kind of excited and then disappointed when Ken took the lead and then lost it.

I'd really like to see Ken and Brad and Frank Spangenberg or someone play a five day tournament.

i loved ken fake throttling the computer monitor while the credits rolled and them putting bunny ears and whatever on it.
our best and brightest still fully capable of thumbing their nose at the future

you really have to read that chat interview with him upthread, I don't think he's really thumbing his nose, he's just naturally a clown. seems to be very quick-witted which would make him really fun to drink with

If this is anything other than hyperbolic, Watson is way way more powerful than we think. But it's probably nonsense:

Within a year, Siegel hopes that "Dr. Watson" will change all of that. Watson is expected to be able to take a patient's electronic medical records, digest them, summarize them for the doctor and point out any causes for concern, highlighting anything abnormal and warning about potential drug interactions.

"It offers the potential to usher in a whole new generation of medicine," Siegel said. "If all Dr. Watson did was allow me to organize electronic medical records and bring to my attention what's most important and summarize it, that would be incredibly valuable to me."

http://www.computerworld.com/s/article/9209899/IBM_s_Watson_could_usher_in_new_era_of_medicine

oh, i love Ken! I read his blog regularly.

if watson CAN live on the cloud and be deployed through speedy voice command through say, PC/PS3/standalone console then IBM has a solid google killer on their hands.
but yeah, i agree that it's probably nonsense

Summarisation/Information Extraction with the reliability required for medical deployment is years away, even a decade. These tasks are orders of magnitude harder than Wikipedia lookup, so if IBM can do this then they have won Technology.

point out any causes for concern, highlighting anything abnormal and warning about potential drug interactions.

this kind of thing is already being done in EMRs

Ok, maybe I missed the point a bit (don't know anything about EMRs). If the input data is already very structured then this reduces to database querying?

yeah, i think the only potentially new thing mentioned in that article is this:

Then when a doctor is considering treating a patient with a particular drug or treatment, they first can ask Watson how that treatment worked on patients with similar diagnoses and backgrounds.

"Watson can ingest information efficiently and rapidly," Siegel said. "It'll have an encyclopedic knowledge and suggest diagnostic and therapeutic possibilities based on databases much larger than one physician can possibly hold in his head.

similar things to this in the works now, but not in such an automated fashion. although it seems that accessing, connecting, and reconciling the databases from hundreds of different hospitals is the real challenge here, moreso than speedy querying/searching.

Thinking about Atul Gawande's articles, seems like a lot of the value here would just be simple checklists, based on peer-reviewed results synthesized by humans. I just can't see people deciding treatment based on unsupervised machine learning algorithms.

Yeah I'm not sure I see the application of Watson's language comprehension (which is the real breakthrough here) on medical diagnosis.

https://chart.googleapis.com/chart?cht=bhs&chco=FF0000|00FF00|0000FF,FFC6A5|DEF3BD|C6EFF7&chs=200x125&chd=s:FOE,elo&chxt=x,y&chxl=1:|It

Jennings writes about the experience:
http://www.slate.com/id/2284721/

Watson has lots in common with a top-ranked human Jeopardy! player: It's very smart, very fast, speaks in an uneven monotone, and has never known the touch of a woman.

seems to be very quick-witted which would make him really fun to drink with

Too bad he's a teetotaler!

i think i remember from his book that he was on a pretty kick-ass bar trivia team tho right? yea itd prob be a blast to bro down w/ him

y'know...if only...

Yeah I'm not sure I see the application of Watson's language comprehension (which is the real breakthrough here) on medical diagnosis.

^^^this is OTM

people rhapsodizing about the glorious future of robotics annoy me, they always massively overstate their cases

maybe if you rephrased your symptoms in the form of a jeopardy question...

if a patient's medical record was really just a mess of free-text, then i can see how it might be useful in combing through for usable data, but that's not the case.

What does Watson's victory mean for humankind? Sounds like something we should ask Watson.

If only I could put it into the form of an answer.

42

i thought it was funny how the ranked answer-candidates that they showed were clearly straight out of watson's internals and not always cleaned up—on one that he didn't get (beat out iirc?), 'reinstate', the answer he would have given was 'reinstate 2'.

also, when did they change the boop-boop-boop-boop-boop-boop-boop-boop-booooop sounds at the beginning, or was that just for the watson-tacular?

i wonder how weird the audience feels when they clap when watson gets a daily double right.

xxp Couple years ago, I think.

lame

I was surprised he was totally stumped on the "Also a Computer Key" category. You'd think that would be right up Watson's alley.

Lots of interesting tidbits here: http://asmarterplanet.com/blog/2011/02/the-watson-research-team-answers-your-questions.html

Couple of surprising things, including the fact that some components are written in Prolog.

very interesting but this argument really annoys me:

'The clues are in English — Brad and Ken’s native language; not Watson’s. Watson must calculate its response in 2-3 seconds and determine if it’s confident enough to buzz in, because as you know, you lose money if you buzz in and respond incorrectly.'

what's the argument

just mean I hate how weird and defensive the guy's getting when confronted w/ the fact that a big part of the victory was just 'robots can press buttons faster than human beings'

http://i.imgur.com/77BFZ.jpg

Ken answers a zillion questions on reddit.
Very funny guy; on Trebek: "He's a riddle wrapped in an enigma wrapped in a Perry Ellis suit."
http://www.reddit.com/r/IAmA/comments/fwpzj/iama_74time_jeopardy_champion_ken_jennings_i_will/

On the awkward moment at the end of an episode:
"Two of you are shell-shocked and pissed, one of you has just realized he's going to have to come back and do it all again after a 10-minute tape break and one of you is slightly drunk and wants to get out of there before the Lakers game starts."

Q: You're a little too funny, did you hire writers with your winnings?
A: Bruce Vilanch is hiding under my desk right now. Unfortunately he's not writing jokes for me, if you know what I mean.

When KJ was in the midst of his run, I couldn't stand him. Now, I kinda love him.

And there are Charlie Sheen references too:

"GET OFF MY BACK TREBEK! Or "Chaim Trebekovitz," which is his real name. Just sayin"

"Never been high--except on a drug called Ken Jennings!"

"5.My Sunday school teacher, when I was a Mormon teen, once memorably advised us that "There's nothing more overrated than sex, and nothing more underrated than a good bowel movement." It totally worked...I don't remember a single other sermon from when I was a kid, but I think about this guy exactly once a day, and then again once a week."

Whoa:

Q: Not really a question, but you came to my school (Escalante Elementary in Utah) I think in 2004-05 ( I was in 6th grade) and talked to us all about your winning Jeopardy! I just wanted to let you know that a bunch of us wanted to sit up front because we thought you'd throw out money since you won so much! Haha...only in the minds of 6th graders...

A: Sorry! Last time I tried to throw hundred dollar bills at a sixth-grader I spent the night in jail.

personally i can't wait because i'm going to edit the watson episodes so that watson keeps giving really obnoxious answers and alex gets pissed while ken smugly presses on

― this is the internet! gifs are the final word! (Z S), Tuesday, February 8, 2011 10:54 PM (3 weeks ago)

also, watson's voice will be 8x louder than either alex, ken or the other contestant's voices

― this is the internet! gifs are the final word! (Z S), Tuesday, February 8, 2011 10:55 PM (3 weeks ago)

please tell me you're still planning to do this...

The chat on reddit is hilarious (whether it's really him or not).

It's really him. http://ken-jennings.com/blog/archives/2614

http://www.bloomberg.com/news/2012-03-05/ibm-s-watson-computer-gets-wall-street-job-one-year-after-jeopardy-win.html

Interesting...I saw a talk by one of the Watson leads last month where the “next big one for us” seemed to be health informatics, but obviously there is $$$ in fancy retrieval and analysis tools for finance. At some point I wonder whether IBM will just be branding all text mining software as "Watson".

I'm surprised Watson didn't just retire on the jeopardy money.
this economy, huh