WATSON on Jeopardy! In February!

Watson being a computer.

http://www.kurzweilai.net/ibm’s-watson-will-compete-on-jeopardy-in-february

IBM’s ‘Watson’ will compete on Jeopardy! in February
December 14, 2010 by Editor
IBM and America’s Favorite Quiz show Jeopardy! today announced that an IBM computing system named “Watson” will compete on Jeopardy! against the show’s two most successful and celebrated contestants — Ken Jennings and Brad Rutter.

The first-ever man vs. machine Jeopardy! competition will air on February 14, 15 and 16, 2011, with two matches being played over three consecutive days.

Watson, named after IBM founder Thomas J. Watson, was built by a team of IBM scientists who set out to accomplish a grand challenge – build a computing system that rivals a human’s ability to answer questions posed in natural language with speed, accuracy and confidence. The Jeopardy! format provides the ultimate challenge because the game’s clues involve analyzing subtle meaning, irony, riddles, and other complexities in which humans excel and computers traditionally do not.

Watson is a breakthrough human achievement in the scientific field of Question and Answering, also known as “QA.” The Watson software is powered by an IBM POWER7 server optimized to handle the massive number of tasks that Watson must perform at rapid speeds to analyze complex language and deliver correct responses to Jeopardy! clues. The system incorporates a number of proprietary technologies for the specialized demands of processing an enormous number of concurrent tasks and data while analyzing information in real time.

Competing against Watson will be two of the most celebrated players ever to appear onJeopardy! Ken Jennings broke the Jeopardy! record for the most consecutive games played by winning 74 games in a row during the 2004-2005 season, resulting in winnings of more than $2.5 million. Brad Rutter won the highest cumulative amount ever by a single Jeopardy! player, earning $3,255,102. The total amount is a combination of Rutter’s original appearance in 2002, plus three Tournament wins: the “Tournament of Champions” and the “Million Dollar Masters Tournament” in 2002 and the “Ultimate Tournament of Champions” in 2005.

The grand prize for this competition will be $1 million with second place earning $300,000 and third place $200,000. Rutter and Jennings will donate 50 percent of their winnings to charity and IBM will donate 100 percent of its winnings to charity.

“After four years, our scientific team believes that Watson is ready for this challenge based on its ability to rapidly comprehend what the Jeopardy! clue is asking, analyze the information it has access to, come up with precise answers, and develop an accurate confidence in its response,” said Dr. David Ferrucci, the scientist leading the IBM Research team that has created Watson. “Beyond our excitement for the match itself, our team is very motivated by the possibilities that Watson’s breakthrough computing capabilities hold for building a smarter planet and helping people in their business tasks and personal lives.”

“We’re thrilled that Jeopardy! is considered a benchmark of ultimate knowledge,” said Harry Friedman, Executive Producer of Jeopardy!. “Performing well on Jeopardy! requires a combination of skills, and it will be fascinating to see whether a computer can compete against arguably the two best Jeopardy! players ever.”

Jeopardy!, the winner of 28 Emmy awards since its syndicated debut in 1984, is in the Guinness Book of World Records for the most awards won by a TV Game Show. The series is the #1-rated quiz show in syndication with nearly 9 million daily viewers. Jeopardy! is produced by Sony Pictures Television, a Sony Pictures Entertainment Company. It is distributed domestically by CBS Television Distribution and internationally by CBS Television International, both units of CBS Corp.

Beyond Jeopardy!, the technology behind Watson can be adapted to solve problems and drive progress in various fields. The computer has the ability to sift through vast amounts of data and return precise answers, ranking its confidence in its answers. The technology could be applied in areas such as healthcare, to help accurately diagnose patients, to improve online self-service help desks, to provide tourists and citizens with specific information regarding cities, prompt customer support via phone, and much more.

Prepping a Machine to Play a Human

This fall, Watson played more than 50 “sparring games” against former Jeopardy! Tournament of Champions contestants in final preparation for its television debut. In addition, Watson has taken and passed the same Jeopardy! contestant test that humans take to qualify to play on the show, giving Jeopardy! producers confidence that the match will be both entertaining and competitive. Highlights of the sparring matches can be viewed and tracked over the next few weeks at www.ibmwatson.com.

https://www.youtube.com/watch?v=FC3IryWr4c8

Very excited!

this rules

DUMPLINGS!

I mean to post that on another thread. Sorry.

Woah...wondering what's inside the box. Presumably the speech-to-text component is good, and they have a dump of Wikipedia/the web/whatever, but I'm very surprised the QA system is competitive with good human contestants, assuming the questions are not completely formulaic (not too familiar with Jeopardy). There are some basic linguistic phenomena that natural language understanding systems just cannot handle.

Man, what does IBM even do these days?

Yeah, seems like they'd be better off having it just play regular people on Jeopardy for a while and forego the whole Ken Jennings thing! Seeing it even sort of manage against vaguely competent players would be impressive enough.

and based on that video they have a ways to go

I am wondering if it'll have problems buzzing in at the proper time. Isn't that something that a lot of former contestants talk about, how there's kind of a peculiar timing that's required?

Yes.

Yeah if a human can buzz in before the whole answer's read out, how is a computer gonna manage that? I mean maybe thats the point here, that its impressive enough to be predictive and whatnot.

That's gotta be the least of their problems!

I figure it might do okay with things like tonight's "A + 4 letters" category - if it can understand the parameters of the category, then it shouldn't be too hard to keep up with the associative/keywordiness of the clues and just skim through the dictionary. And some of them are narrow enough trivia that I could imagine it working - like, y'know, "In winter of such and such year in such and such city, the first one of THESE was held" - - basically, if you had the speed of a computer to Google the year, the city, and Winter, you'd be really close to the answer.

xpost

Yeah if a human can buzz in before the whole answer's read out

No, the way it works is that you have to wait until Alex finishes reading the clue, and if you buzz in too early, you get locked out for like a quarter of a second. (On the first Trebek season, in 1984-85, you could buzz in whenever, but it didn't make for particularly good TV.) So presumably this won't be an issue -- Watson will be working with the same complete set of information as everyone else.

I guess I am wondering about the timing, though. If Watson knows the right answer, will it always be able to buzz in at exactly the appropriate moment? (Also, does it ever wait and "think" about it before buzzing in?)

Ah right I didnt know that - yeah that helps. The YT clip was great. This is so cool.

Judgment Day next week! ZOMG!!! HUMANS PREPARE TO DIE!!!!!!!!!!!

personally i can't wait because i'm going to edit the watson episodes so that watson keeps giving really obnoxious answers and alex gets pissed while ken smugly presses on

also, watson's voice will be 8x louder than either alex, ken or the other contestant's voices

lol z_s wins

kind of in this style:
https://www.youtube.com/watch?v=O9f1zM7I8Z8

my goal is to get a video up in the middle of next week and hopefully get a million views for people searching for "Jeopardy Watson Computer"

watson is killin all yall jive turkeys

Watson!!!

how did our boy do

seems kinda unfair that he doesnt get a text file of an opponents answer if they get it wrong

also this sorta felt like a long commercial for ibm

seems kinda unfair that he doesnt get a text file of an opponents answer if they get it wrong

― johnny crunch, Monday, February 14, 2011 8:49 PM (3 minutes ago)

haha yeah alex is like "uh no...ken already guessed that"

yeah but it's not like he "hears" that either!

what is the score?

watson 5000
brad 5000
ken ~2000

man ken really doesn't have it anymore

did it air?

yeah it was on today, i guess it's just like a practice round or something tho?

its gonna be 2 complete games spread over 3 days of shows iirc

ken threw the practice round so he could bank in las vegas

best part was watson found the daily double on his first category pick on a full board i mean c'mon man

yeah that was ridic

watson went hard in the paint

big up

omg @ Z S upthread -- fantastic

this was waaay more fun to watch having actually been on jeopardy recently

i'm still unclear as to how watson knows when to ring in: presumably they send it some kind of signal, even though it can't see the lights? getting used to trebek's cadence was definitely the hardest part of it for me; as i'm sure jaymc can attest, all the onstage practice is hosted by a dude who talks nothing at all like trebek

One thing that is clear is that Brad is really ridiculously good and would probably have won a bunch of games in row (okay maybe not 75+) if he'd be allowed to back in his day.

Was kind of feeling bad for skintubes in the beginning, but it seemed like they were getting used to the way Watson rang in.

i'm still unclear as to how watson knows when to ring in

Yeah. Maybe some voice recognition software? There has to be some analog element to it since Brad sometimes was buzzing in before W even if W's confidence level met its threshold.

I'm betting that Watson out-buzzed both of them quite a few times.

Yeah, and the other guys got in before Watson, too.

Well, I mean, the humans can only buzz in at a certain official time too - there's a light that switches on or something, right? Before which time, if you buzz in, it locks you out for the first second or two of the actual "open buzz-in" time. So it's not just like, "when Alex is done reading you can buzz in" - there's a hard, 0 or 1 style limit that would be easily converted to a digital thing.

Somebody must be assembling all the times Watson was wrong and figuring out what they have in common, yes? It doesn't seem like obscurity of information was the issue, so it has to be a wording/complexity/subtlety of meaning issue?

I was mildly annoyed that "Maxwell's Silver Hammer" was accepted for what, syntactically, should have been Maxwell," but Alex accepts that stuff all the time from human contestants, so hey.

The only bummer with seeing the confidence level score is that the player at home (aka me) has an even shorter window to come up with the answer.

hahaha that was bothering me so much! /dork

when i saw this thread on new answers the first few times i was like "there's someone on ilx named Watson? what display name do they post under?"

then when i turned on Monday's show and saw the graphics with the three answers and the percentages next to them at the bottom of the screen, i went "woah, what's going on, is this a Who Wants To Be A Millionaire-style special edition of Jeopardy?". it took me a good 30 seconds to put 2 and 2 together.

once he masters jeopardy watson'll start posting on ilx

Yeah. Maybe some voice recognition software? There has to be some analog element to it since Brad sometimes was buzzing in before W even if W's confidence level met its threshold.

Trebek said that they send Watson a text file at the same time the question is read. Kinda wish it was voice recognition software, actually.

seems kinda unfair that he doesnt get a text file of an opponents answer if they get it wrong

― johnny crunch, Monday, February 14, 2011 8:49 PM (3 minutes ago)

Was wondering about this. Ken guessed, "What is the 20s?" Watson then guessed, "What is 1920s?" Is it possible that Watson knew Ken's response but thought its own response was different?

whoa did I miss this? Or will it re-air?

https://www.youtube.com/watch?v=4PSPvHcLnN0
https://www.youtube.com/watch?v=CtHlxzOXgYs

Need to watch this, the engineering behind it is pretty close to what I do every day. Didn't realise they get a text feed, that makes it a bit easier and reduces everything to a very fast search problem. If you could Google/Wikipedia really quickly, how difficult would the questions be?

The other important component in question answering is recognising the "answer type" - is the questioner looking for a person, a place, a thing, etc.? This is usually clear from the beginning of the sentence, but I'm not sure whether Jeopardy! mixes this up.

it's interesting that, according to Trebek, the computer *only* has 75 trillion bytes of RAM, which works out to just over 68 terabytes. that's hardly a mindblowing figure when 2 terabyte hard drives sell for about $100 apiece. on the other hand, I'm sure most or all of the data fed to Watson is text-based, and text doesn't take up a lot of space compared to, say, sound files. since all the text from all the articles on English-language Wikipedia amounts to about 27 gigabytes, then Watson has a capacity of over 2,500 Wikipedias.

RAM is different than storage space, usually RAM costs about $20/gb so that is quite expensive.

Watching the clips again, it almost seems like Ken (at least) switched to a strategy of just buzzing in like crazy even if it took him a second to find the answer, knowing that 90% of the time Watson is going to buzz fast and be right. Like, he definitely found "Julia" in his head after buzzing in, but obviously WATSON can handle song lyrics pretty easily.

This was interesting.

xp that clue particularly had 2 be really easy for watson, all you need is 'john lennon' & 'mother' -- not sure that i recall a clue where i was impressed watson could figure it out

89% WHAT IS MAMA DON'T GO DADDY COME HOME
86% WHAT IS YOU HAD ME
44% WHAT IS I NEVER HAD YOU

Watching the clips again, it almost seems like Ken (at least) switched to a strategy of just buzzing in like crazy even if it took him a second to find the answer, knowing that 90% of the time Watson is going to buzz fast and be right. Like, he definitely found "Julia" in his head after buzzing in, but obviously WATSON can handle song lyrics pretty easily.

― Doctor Casino, Tuesday, February 15, 2011 12:14 PM (3 minutes ago) Bookmark Suggest Ban Permalink

iirc wasn't it well known during his original winning streak that he had really good buzzer timing and would often buzz in whether or not he knew the answer and just knew the answer often enough that the strategy worked?

http://www.nytimes.com/2010/06/20/magazine/20Computer-t.html

havent watched the show but last summer nyt had a long article on 'watson' thats p good

xxxpost They were also generally really easy questions - - - not atypically easy for first-round Jeopardy, but not champion-level difficulty either. Of course, if you're WATSON, obscure trivia may not actually be harder to find than regular trivia, right? The real issue is seeing past the riddle-like language to find the keywords that matter as opposed to the ones that are just flavor.

I mean, it's pretty cool that he sees the list of people Voldemort's killed and understands the category's about villains (I guess because of "APB") and comes up with Voldemort and not, you know, Dumbledore or Harry Potter. We're a long way from "Computer! Tell me, who killed Charity Burbage?" but I do think it's kind of neat.

maybe next they can reverse engineer Ask Jeeves to answer in the form of a question

I mean, it's pretty cool that he sees the list of people Voldemort's killed and understands the category's about villains (I guess because of "APB") and comes up with Voldemort and not, you know, Dumbledore or Harry Potter.

Wait, I could be misremembering, but I thought that was one that Watson didn't buzz in on, but its best guess (percentage-wise) was Harry Potter.

hahaha oops i think you're right

I was wondering, does Watson even factor in the category title at all? That seems like it could be one of the trickier things to get a computer to figure out, but it is pretty crucial for certain categories. So does it maybe try to figure out exactly what the category is about, and if it can't, just answer the questions as if they were categoryless?

Brad is just so good I kind of think of him as a computer specially designed to play Jeopardy.

Could be some kind of weighted thing surely? Like WATSON pulls up a few dozen or a few hundred general associations for the category title, and if it has a guess for a clue that doesn't ring any of those keywords, it rejects that guess straight out? Or it gets a big penalty on certainty? I mean, you probably don't need the category at all for Beatles song lyrics, but when the category is DECADES it's kind of crucial to know that you're guessing a decade.

Brad looking a bit Jon Hamm-ish, IMO.

right, or, say 4 letter words. but there has to be some categories where it just throws up it hands, doesn't have much of a clue what it's about, right?

i just hope this opens the door for better gimmicky jep episodes. im sorry but no one needs the teen tournament

teen tourny is good makes me feel smart

Answering easy questions doesn't make me feel smart.

iirc wasn't it well known during his original winning streak that he had really good buzzer timing and would often buzz in whether or not he knew the answer and just knew the answer often enough that the strategy worked?

I actually read Ken Jennings' book and he pretty much admits that this was a huge factor in his streak. He says that for most clues two or all three contestants knew the answer, but he was just a little quicker and knew the way the timing worked. Great read BTW

I never read his book but I looked at his blog for a while, he seems like a genuinely good dude

They were also generally really easy questions - - - not atypically easy for first-round Jeopardy, but not champion-level difficulty either. Of course, if you're WATSON, obscure trivia may not actually be harder to find than regular trivia, right? The real issue is seeing past the riddle-like language to find the keywords that matter as opposed to the ones that are just flavor.

Right, this for me is the most interesting part of this experiment. There are lots of puns in Jeopardy clues and categories and I wonder if/how a computer can figure them out. Either way I'm pretty stoked for this..

Yeah, definitely! I'm not at all surprised that a computer can search-engine from keywords really fast and gin up a probable answer, but being able to sift through the riddle-like Jeopardy answers to figure out what the keywords ARE is pretty cool. I'd like to see more of the graphics that describe how WATSON gets there, the ones that show it sifting through the keywords and reducing them down to the likely answers or whatever...that's cool.

Anyone remember a website called ASK HAL or something like that, where you typed stuff at this keyword-based AI on the Internet, and it strung together answers based on stuff other people had said to it? Occasionally it would get junked up with huge chunks of text people had pasted in, but every so often it'd be just frighteningly lucid and clear. Not that it "knew" what it was saying, but it basically was responding in a logically appropriate way to what you were typing. Or so it seemed to me in 1998 or whenever.

I got a kick out of http://www.20q.net/ back in the day.

seems kinda unfair that he doesnt get a text file of an opponents answer if they get it wrong

Heard an interview with Ken and Brad yesterday where they said someone would actually be feeding Watson their responses during the game. Maybe it just didn't get Ken's "What is the 20s?" in time? Or what jaymc said.

btw here are all the questions Watson got wrong, interesting imo

https://spreadsheets.google.com/lv?hl=en&hl=en&key=tth_jhM8vyBAuogqHllHmHQ&type=view&gid=0&f=true&colid0=27&filterstr0=No&sortcolid=-1&sortasc=true&rowsperpage=250

watson clearly has a low opinion of ken & thought he guessed 20-29 a.d.

what time will this be on today?

it's syndicated, so depends where you live

7pm EST

Not necessarily forks - it's 7 in NY where my folks live but 7:30 here in Boston.

You have to check yr local listings.

i stand corrected
corrected by SCIENCE

Yeah, and 3:30 PM in Chicago, which means I never get to watch the show. (I found that YouTube this morning.)

What? That's absurd. It is clearly an evening program.

It was great when I was a kid and I could watch it first thing after school, but yeah kind of dumb now.

ha it seems weird to me that it's shown in the evening elsewhere. jaymc you know it's shown at like 6pm Saturdays, right? i was never able to figure out in what sequence the Saturday episodes run, cause the Monday afternoon one is always the next in sequence after the Friday one.

When I was growing up we'd always eat dinner as soon as it finished and now i often eat dinner while watching it. Having it in the afternoon would just be all wrong! lol It never even occurred to me it might be a day thing in other places.

yeah, i think of it a show for sharp retirees, housewives, and stoners (via 3:30 pm).

ok i'm not crazy then because i could have sworn i remembered it being on at 730 when i was a kid - it's on at 7 up here and my roommate insisted it's on at 7 everywhere and i was just remembering wrong

7:30 on a lot of the west coast, like always, after wheel of fortune

lol, yesterday i asked my friend if he'd seen the episode and he goes "yeah we watched it at work" and i was like "huh..." and then i had this same reaction of befuddlement as he told me that it comes on at 3:30 in STL

3:30 is an evening program for seniors

Trebek said that they send Watson a text file at the same time the question is read. Kinda wish it was voice recognition software, actually.

― Tyler/Perry's "Dude (Looks Like a Lady)" (jaymc)

Right but don't you get to read the question on your own the exact time Watson gets to read the question? I mean, is the screen readable when you are on the show?

so now that it's obvious that humans can't compete against this machine, can i just say, "congratz on building a machine that can win a trivia game plz work on something important now"

Why do you care how IBM invests its resources? Anyway, it's pretty easy to see how this could be applied.

i just think it's silly. it was interesting (to me at least) for the first episode but once it became clear that no one could compete with the computer it became vastly less interesting. also, maybe i misunderstand the technology but is this really a huge development/breakthrough? it seems like this is pretty standard algorithmic tech (and certainly deep blue became much less interesting after it won the first time too) and not totally different from the hundreds of times a day i use google.

It isn't a breakthrough. The nuances of language are an ongoing problem with respect to data mining, and this is a fun benchmark along the way to better technology.

and this is a fun benchmarkmarketing goldmine along the way to better technology

Very true.

http://26.media.tumblr.com/tumblr_lgokh0zWkf1qztqsao1_500.png

HOST: This trusted friend was the first non-dairy powdered creamer. Watson?

WATSON: What is milk? [Research team in audience laughs]

HOST: No! That wasn’t wrong, that was really wrong, Watson.

Right but don't you get to read the question on your own the exact time Watson gets to read the question? I mean, is the screen readable when you are on the show?

Yeah, it is -- although it felt a little far away for me (maybe just my failing eyesight), so I was glad I had multiple sensory cues.

jaymc you know it's shown at like 6pm Saturdays, right?

Yeah. I never remember to watch it, though. And then when I do, it seems like there's always a sporting event that's preempting it.

It isn't a breakthrough. The nuances of language are an ongoing problem with respect to data mining, and this is a fun benchmark along the way to better technology.

I think a lot of the cool things about Watson are going to be unglamorous engineering aspects, e.g. how to index all the stored data in a way that facilitates fast real-time search...as for the question answering itself, I can write down a bunch of methods they might be using but going from there to a functioning system is going to involve a lot of subtleties.

Ultimately, the main point of Watson is publicity for IBM. It also gives me something besides Google Translate that I can mention when people ask me what natural language processing is for.

so obvious that there was no way it could've possibly gotten the final jeopardy question. most of the clues played to its strengths, tho — I'm p.impressed

i mean, watson, the category was u.s. cities bro

hope they have a before & after category 2morrow

Giant search engine rings in faster than humans, story at 11 (3:30 in Chicago).

It also gives me something besides Google Translate that I can mention when people ask me what natural language processing is for.

translators are what excite me about this.

not that there won't be something pretty miserable about all the world's tourists walking around talking imperiously through universal-translator boxes, ordering croissants and disneyland tickets in the same robot voice every time.

so did the computer lose the money re: that pict or what?

During the credits and you see the contestants talking to Trebec while the theme music rolls and Jennings is saying "Watson's a bitch". Trebec's says "don't I know it" and Watson is all like "(beepboopbloopbloopbloopeeeeeoooooo...)"

It was impressive but honestly it looked like Ken and Brad knew most of the answers and just couldn't buzz in on time. Whatever was giving Watson the signal to buzz in seemed to react faster than a human could. You could see a look of frustration on Ken's face as he was pretty clearly hitting the buzzer on nearly every question. I was able to get about 35-40% of them, I would imagine between Ken and Brad they could have nearly cleared the board if there wasn't someone buzzing in with robotic precision at the earliest possible moment. Combined with everything else it really just seemed like one long commercial for IBM but I guess this has really been the first real interesting Jeopardy since 2004, so more power to them.

natural language processing is one of the holy grails of artificial intelligence innit - unfortunately it still looks like the way there is through brute force...but maybe that's how the brain works too

funny watson 'cameo' on conan last night

yeah i enjoyed that

i loved when andy said 'i think i might have to take it to the voiceover booth' cuz it got a real genuine laugh from conan

i think a more fair competition would be to have a human on the buzzer taking cues from Watson's color rather than having the fucking thing autoanswer

Awesome interview with Ken Jennings about this here:
http://live.washingtonpost.com/jeopardy-ken-jennings.html

Q.if you won....
If you are the winner, would you be willing to sit with the Watson designers to improve the machine even further? If so, what would you suggest?

– February 11, 2011 6:28 PM Permalink
A.Ken Jennings :
The Watson team told me two things after the match: that the idea for Watson was born after watching my 2004 streak on Jeopardy, and that they watched LOTS of tape of me while honing its skills. "There's a lot of you in Watson," one guy said. So I already feel like the Dr. Frankenstein here. If it goes amuck and kills humanity and stuff so sorry lolz my bad!

it struck me watching this on tuesday (i didn't see monday's episode) that the only thing this is definitively proving is that a computer can hit a buzzer faster than a computer can. the score isn't reflecting what the computer knows and what the humans don't know. it's reflecting buzzer skills.

buzzer skills are pretty much what jeopardy ultimately is tbqh

also, i wish alex had been sterner in correcting watson's mispronunciations! and i really really wish they had done the normal segment where alex interviews the contestants. (or did they do that the first day and they just dropped it yesterday?)

yea tbf im sure a lot of "smarter" contestants have lost by way of being out-buzzered

yea they did the talk 2 contestants on monday

but did he interview watson?

it struck me watching this on tuesday (i didn't see monday's episode) that the only thing this is definitively proving is that a computer can hit a buzzer faster than a computer can. the score isn't reflecting what the computer knows and what the humans don't know. it's reflecting buzzer skills.

― fact checking cuz, Wednesday, February 16, 2011 10:57 AM (16 minutes ago) Bookmark

well...if the computer didn't know the answer, it wouldn't get the score and one of the other guys gets a shot, so there is a little more to it than the buzzer.

Right, that's something that Ken brings up a lot, but still I don't think anyone can beat a mechanical buzzer any more than a person can outrace a train on foot. Hence why it seemed like a long commercial for IBM. Even so the thing is simply amazing; if you saw the questions asked it definitely did more than just "fast wikipedia search". Of course if the thing really requires 68 TB of RAM to run it's going to be a while before we can use this in practical ways.

well...if the computer didn't know the answer, it wouldn't get the score and one of the other guys gets a shot, so there is a little more to it than the buzzer.

well yeah, of course, so it's proving that the computer knows most of the answers. but it's pretty clear to me that the humans know most of the answers too. hell, i knew most of the answers yesterday, and i'm no ken jennings. but because of the buzzer, the computer basically gets first crack at everything, so of course it's gonna win. it's totally impressive and fascinating anyway, but as a science experiment or demonstration, it just seems kinda off to me.

is there one more day of this? I'm actually going to be home this afternoon.

Yup. The final final is today.

I'll take "integers divided by zero" for $1200, Alex

**Watson explodes**

i'm sure they could, and probably have, done tests or exhibitions for Watson in which its percentage of correct answers to questions is measured against that of humans on an even-playing field. but if they did that on Jeopardy, it wouldn't be Jeopardy, and i'd rather its buzzer be too fast than it be too slow and you don't even get to see how well it could do.

watson has basically already won this; either human would need to outscore it by over 20K in one game and i've never seen that happen.

i guess it could happen; the humans seem to know that to have any shot they would have to play very agressive (buzzing in and attempting to figure out the answer 3 seconds later, betting big on FJ and DDs) and I guess I can see one of them winning thanks to one or two confusing categories. 99% sure it won't though

best part was watson found the daily double on his first category pick on a full board i mean c'mon man

― johnny crunch, Monday, February 14, 2011 8:07 PM (2 days ago) Bookmark Suggest Ban Permalink

this is the worst time to find the daily double though.

well, right, but it keeps an opponent from hitting it. plus i like the idea that watson couldnt really grasp that level of gameplay strategy & honed right to it

don't really get how Watson could have intentionally found the DD on its first guess - wasn't it probably just a coincidence?

i guess it could happen; the humans seem to know that to have any shot they would have to play very agressive (buzzing in and attempting to figure out the answer 3 seconds later, betting big on FJ and DDs) and I guess I can see one of them winning thanks to one or two confusing categories. 99% sure it won't though

Plus, there's a divide-and-conquer problem - - not that the questions are hard but certainly Brad and Ken both have first-rate Jeopardy buzzer timing...putting BOTH of them against WATSON means that they split the pickups. Would have been better as just WATSON versus Ken, but I could see where that would be kinda weird, just like further re-coronation of Ken as the "greatest ever" etc.

Keep thinking of this Watson and this Alex
http://www.lookandlearn.com/blog/images/A005753.jpg
but maybe I'm not the only one.

Ken Jennings :

Daily Doubles aren't distributed randomly...basically, it had thousands of old Jeopardy games in its head and knew where to look. And got very lucky.

Most players go top-down, but some hunt for Daily Doubles like Watson. Brad Rutter calls this an arms race: if one player does it, you have to join in to keep up.

The humans look pissed

https://www.youtube.com/watch?v=Ga0M_kQVzsI

hey I just realized something; why don't audience members on shows like Who Wants to be a Millionaire just tip off the contestants using smartphones?

presumably you're not allowed to have your cell phone on you during the show?

yeah but do they pat people down? or host them in a building encased with aluminum? i'm genuinely curious. i'd assume it's impossible because nobody's done it yet (or rather, been caught)

RIP humanity. (Good riddance!)

I still think there has to be an analog element,to when Watson buzzes in, because otherwise, it'd NEVER lose to a puny human whenever its confidence was high enough. Maybe it has something to do with the guy who alerts contestants to the open-buzzing window (see the NYT article above). That said, it was clearly steamrolling the other guys when it came to buzzing in on night 2.

hey I just realized something; why don't audience members on shows like Who Wants to be a Millionaire just tip off the contestants using smartphones?

http://en.wikipedia.org/wiki/Charles_Ingram#Who_Wants_to_Be_a_Millionaire.3F_affair

is there any way to run Watson's voice synth on my home computer without racks of servers?

apparently not:
http://www-03.ibm.com/innovation/us/watson/research-team/speech.html

Steven Pinker:

I don’t rule out the possibility that some components of Watson could both provide insight into human cognition and lay the groundwork for more sophisticated artificial intelligence applications, such as natural language processing (the fancy term for understanding human languages like English, as opposed to computer languages). On the other hand, when a system is designed to meet a highly specific challenge like playing Jeopardy, and one where the reputations of the designers are on the line, there will be enormous pressure to tailor the system to succeeding at that challenge by any means whatsoever, including kludges that are specific to the rather peculiar requirements of the game of Jeopardy.

http://www.newyorker.com/online/blogs/newsdesk/2011/02/what-would-watsons-victory-mean.html

kinda crushin on Bhuvana Ramabhadran, not least cuz i'd love to carry that last name as a hyphenate
http://www-03.ibm.com/innovation/us/watson/images/what-is-watson/research-team/speech/bhuvana-ramabhadran.jpg

is she some kind of child genius?

Six people on the Speech Team, and they still couldn't prevent Watson from pronouncing "Brute" (in the category "Etude Brute") with one syllable.

aw, she let her mother sit in for her on the staff picture at the jeopardy podium!
http://www.research.ibm.com/deepqa/speech_team.shtml

kinda wanna make her into a meme, y cuz she is fly
http://www.signalprocessingsociety.org/uploads/sltc/MemberPics/bhuvana.jpghttp://www.signalprocessingsociety.org/uploads/sltc/MemberPics/bhuvana.jpghttp://www.signalprocessingsociety.org/uploads/sltc/MemberPics/bhuvana.jpg

aw, she let her mother sit in for her on the staff picture at the jeopardy podium!

I think that's a mistake. The woman pictured is Bhavani Iyer, of the Systems Team:
http://www.research.ibm.com/deepqa/systems_team.shtml

damn, your .xls skills is hella speedy

almost...suspiciously so...

http://www.freevistawallpaper.com/wp-content/uploads/hal-9000-vista-wallpaper.jpg

Open the Ramabhadran Bay Doors Please HAL

that jennings interview is glorious

Six people on the Speech Team, and they still couldn't prevent Watson from pronouncing "Brute" (in the category "Etude Brute") with one syllable.

Maybe, but I'm still kind of amazed that it knows when to pronounce things in a Frenchy way (e.g. "Jean Valjean")!

lol that website spelled her name wrong even though she literally wrote it out in that picture

No bump for yesterday's episode? Ken actually jumped out to a big lead in the first round; the machine seemed pretty stumped for a bunch of questions. Of course Watson picked it up in double Jeopardy and then nailed FJ; still, it looked to me like Ken knew the answer to almost every question but couldn't buzz in on time. "I for one welcome our new computer overlords" - priceless

yeah i actually got kind of excited and then disappointed when Ken took the lead and then lost it.

I'd really like to see Ken and Brad and Frank Spangenberg or someone play a five day tournament.

i loved ken fake throttling the computer monitor while the credits rolled and them putting bunny ears and whatever on it.
our best and brightest still fully capable of thumbing their nose at the future

you really have to read that chat interview with him upthread, I don't think he's really thumbing his nose, he's just naturally a clown. seems to be very quick-witted which would make him really fun to drink with

If this is anything other than hyperbolic, Watson is way way more powerful than we think. But it's probably nonsense:

Within a year, Siegel hopes that "Dr. Watson" will change all of that. Watson is expected to be able to take a patient's electronic medical records, digest them, summarize them for the doctor and point out any causes for concern, highlighting anything abnormal and warning about potential drug interactions.

"It offers the potential to usher in a whole new generation of medicine," Siegel said. "If all Dr. Watson did was allow me to organize electronic medical records and bring to my attention what's most important and summarize it, that would be incredibly valuable to me."

http://www.computerworld.com/s/article/9209899/IBM_s_Watson_could_usher_in_new_era_of_medicine

oh, i love Ken! I read his blog regularly.

if watson CAN live on the cloud and be deployed through speedy voice command through say, PC/PS3/standalone console then IBM has a solid google killer on their hands.
but yeah, i agree that it's probably nonsense

Summarisation/Information Extraction with the reliability required for medical deployment is years away, even a decade. These tasks are orders of magnitude harder than Wikipedia lookup, so if IBM can do this then they have won Technology.

point out any causes for concern, highlighting anything abnormal and warning about potential drug interactions.

this kind of thing is already being done in EMRs

Ok, maybe I missed the point a bit (don't know anything about EMRs). If the input data is already very structured then this reduces to database querying?

yeah, i think the only potentially new thing mentioned in that article is this:

Then when a doctor is considering treating a patient with a particular drug or treatment, they first can ask Watson how that treatment worked on patients with similar diagnoses and backgrounds.

"Watson can ingest information efficiently and rapidly," Siegel said. "It'll have an encyclopedic knowledge and suggest diagnostic and therapeutic possibilities based on databases much larger than one physician can possibly hold in his head.

similar things to this in the works now, but not in such an automated fashion. although it seems that accessing, connecting, and reconciling the databases from hundreds of different hospitals is the real challenge here, moreso than speedy querying/searching.

Thinking about Atul Gawande's articles, seems like a lot of the value here would just be simple checklists, based on peer-reviewed results synthesized by humans. I just can't see people deciding treatment based on unsupervised machine learning algorithms.

Yeah I'm not sure I see the application of Watson's language comprehension (which is the real breakthrough here) on medical diagnosis.

https://chart.googleapis.com/chart?cht=bhs&chco=FF0000|00FF00|0000FF,FFC6A5|DEF3BD|C6EFF7&chs=200x125&chd=s:FOE,elo&chxt=x,y&chxl=1:|It

Jennings writes about the experience:
http://www.slate.com/id/2284721/

Watson has lots in common with a top-ranked human Jeopardy! player: It's very smart, very fast, speaks in an uneven monotone, and has never known the touch of a woman.

seems to be very quick-witted which would make him really fun to drink with

Too bad he's a teetotaler!

i think i remember from his book that he was on a pretty kick-ass bar trivia team tho right? yea itd prob be a blast to bro down w/ him

y'know...if only...

Yeah I'm not sure I see the application of Watson's language comprehension (which is the real breakthrough here) on medical diagnosis.

^^^this is OTM

people rhapsodizing about the glorious future of robotics annoy me, they always massively overstate their cases

maybe if you rephrased your symptoms in the form of a jeopardy question...

if a patient's medical record was really just a mess of free-text, then i can see how it might be useful in combing through for usable data, but that's not the case.

What does Watson's victory mean for humankind? Sounds like something we should ask Watson.

If only I could put it into the form of an answer.

42

i thought it was funny how the ranked answer-candidates that they showed were clearly straight out of watson's internals and not always cleaned up—on one that he didn't get (beat out iirc?), 'reinstate', the answer he would have given was 'reinstate 2'.

also, when did they change the boop-boop-boop-boop-boop-boop-boop-boop-booooop sounds at the beginning, or was that just for the watson-tacular?

i wonder how weird the audience feels when they clap when watson gets a daily double right.

xxp Couple years ago, I think.

lame

I was surprised he was totally stumped on the "Also a Computer Key" category. You'd think that would be right up Watson's alley.

Lots of interesting tidbits here: http://asmarterplanet.com/blog/2011/02/the-watson-research-team-answers-your-questions.html

Couple of surprising things, including the fact that some components are written in Prolog.

very interesting but this argument really annoys me:

'The clues are in English — Brad and Ken’s native language; not Watson’s. Watson must calculate its response in 2-3 seconds and determine if it’s confident enough to buzz in, because as you know, you lose money if you buzz in and respond incorrectly.'

what's the argument

just mean I hate how weird and defensive the guy's getting when confronted w/ the fact that a big part of the victory was just 'robots can press buttons faster than human beings'

http://i.imgur.com/77BFZ.jpg

Ken answers a zillion questions on reddit.
Very funny guy; on Trebek: "He's a riddle wrapped in an enigma wrapped in a Perry Ellis suit."
http://www.reddit.com/r/IAmA/comments/fwpzj/iama_74time_jeopardy_champion_ken_jennings_i_will/

On the awkward moment at the end of an episode:
"Two of you are shell-shocked and pissed, one of you has just realized he's going to have to come back and do it all again after a 10-minute tape break and one of you is slightly drunk and wants to get out of there before the Lakers game starts."

Q: You're a little too funny, did you hire writers with your winnings?
A: Bruce Vilanch is hiding under my desk right now. Unfortunately he's not writing jokes for me, if you know what I mean.

When KJ was in the midst of his run, I couldn't stand him. Now, I kinda love him.

And there are Charlie Sheen references too:

"GET OFF MY BACK TREBEK! Or "Chaim Trebekovitz," which is his real name. Just sayin"

"Never been high--except on a drug called Ken Jennings!"

"5.My Sunday school teacher, when I was a Mormon teen, once memorably advised us that "There's nothing more overrated than sex, and nothing more underrated than a good bowel movement." It totally worked...I don't remember a single other sermon from when I was a kid, but I think about this guy exactly once a day, and then again once a week."

Whoa:

Q: Not really a question, but you came to my school (Escalante Elementary in Utah) I think in 2004-05 ( I was in 6th grade) and talked to us all about your winning Jeopardy! I just wanted to let you know that a bunch of us wanted to sit up front because we thought you'd throw out money since you won so much! Haha...only in the minds of 6th graders...

A: Sorry! Last time I tried to throw hundred dollar bills at a sixth-grader I spent the night in jail.

personally i can't wait because i'm going to edit the watson episodes so that watson keeps giving really obnoxious answers and alex gets pissed while ken smugly presses on

― this is the internet! gifs are the final word! (Z S), Tuesday, February 8, 2011 10:54 PM (3 weeks ago)

also, watson's voice will be 8x louder than either alex, ken or the other contestant's voices

― this is the internet! gifs are the final word! (Z S), Tuesday, February 8, 2011 10:55 PM (3 weeks ago)

please tell me you're still planning to do this...

The chat on reddit is hilarious (whether it's really him or not).

It's really him. http://ken-jennings.com/blog/archives/2614

http://www.bloomberg.com/news/2012-03-05/ibm-s-watson-computer-gets-wall-street-job-one-year-after-jeopardy-win.html

Interesting...I saw a talk by one of the Watson leads last month where the “next big one for us” seemed to be health informatics, but obviously there is $$$ in fancy retrieval and analysis tools for finance. At some point I wonder whether IBM will just be branding all text mining software as "Watson".

I'm surprised Watson didn't just retire on the jeopardy money.
this economy, huh