Man vs App on Jeopardy tonight.

Moonliner · 02-14-2011, 11:09 AM

Tonight is the debut of "Watson" the Jeopardy playing app.

Quote:

Originally Posted by News

Today, tomorrow, and the 16th are the fateful days: IBM's Watson supercomputer will go head to head with Brad Rutter and Ken Jennings in this silly, human-devised game we call Jeopardy. It promises to be some kickass TV, at the very least, and a historic event if Watson can prevail over his fleshy competition.

Here is video of the practice round.

Ghoulish Delight · 02-14-2011, 11:37 AM

Read this about it the other day, written by Bob Harris, 8-time Jeopardy winner and participant in Jeopardy Masters Tournament. I think it's a good summation of what the event "means", if anything.

Ghoulish Delight · 02-14-2011, 09:38 PM

So here's something I've been trying to figure out, but haven't seen anything that explains it. How does Watson understand what the categories mean. Especially the punny type categories. Does it have some definition of the kinds of puns and clue structures that Jeopardy has? Does it try to figure it out from the title? Or does it just ignore it and go purely from question content.

Judging from what we saw tonight, I'm guessing option C. Compare Watson's performance in the "APB" category to the decades category. It kicked butt in the APB category since, while the clue structure was weird, it was chock full of keywords and had very straight forward answers. Just plug in the keywords, voila, answer. Whereas the decade category, just plugging in the keywords that are in the clue would perhaps bring up a bunch of dates, but wouldn't lead directly to answering in the form of a decade (and indeed, on a few of them, the Watson display showed it wasn't even considering a decade as an answer).

So, if that's the case, I'm more impressed that it's really trying to figure out the KINDS of answers that it should be looking for, rather than just being fed the category definitions. Nifty!

And they never show enough detail on the server racks on these things. What kind of storage are they using? What about networking. I presume the servers are clustered, and using 1GbE, but is there a separate storage network. Fibre channel? iSCSI? Infiniband? (hahahaha.........okay, I'm the only one who thinks that's funny)

lashbear · 02-14-2011, 10:29 PM

Infiniband!!! Bwah hahahahaha !

Infiniband?

Moonliner · 02-15-2011, 07:29 AM

Quote:

Originally Posted by Ghoulish Delight

And they never show enough detail on the server racks on these things. What kind of storage are they using? What about networking. I presume the servers are clustered, and using 1GbE, but is there a separate storage network. Fibre channel? iSCSI? Infiniband? (hahahaha.........okay, I'm the only one who thinks that's funny)

Here is the best source I've seen so far on the underlying hardware.

Assuming it's not a typo, Watson has 4TB of storage (the same as my home PC) and 16TB of memory.

Quote:

Originally Posted by SomeGuy

Watson is made up of ninety IBM POWER 750 servers, 16 Terabytes of memory, and 4 Terabytes of clustered storage. Davidian sontinued, “This is enclosed in ten racks including the servers, networking, shared disk system, and cluster controllers. These ninety POWER 750 servers have four POWER7 processors, each with eight cores. IBM Watson has a total of 2880 POWER7 cores.”

Makes sense I suppose, putting everything is RAM is the only way to get the response time you need. I bet it takes a serious amount of time to boot Watson from a cold start.

Moonliner · 02-15-2011, 07:35 AM

WTF! The final jeopardy question was leaked, I'm thinking the fix is in.

Spoiler:

Gn2Dlnd · 02-15-2011, 03:38 PM

Sorry, your question must be phrased in the form of an answer.

Cadaverous Pallor · 02-16-2011, 01:21 PM

This has been so interesting! Watson is killing them. I think the buzzer thing just wasn't calibrated well to allow them to get in.

Watching what Watson gets wrong is so illuminating. Seems the quiz writers put in some extra effort on the Final Jeopardy question and succeeded in flummoxing Watson. Even so, his interestingly particular betting ability kept him out of trouble. The alternate answers chart on the screen including confidence in answers makes this even more fun.

It cracks me up to see the computer scientists in the audience applauding their baby.

Alex · 02-16-2011, 01:37 PM

Well, unless they built in an uncertainty factor (and I haven't been paying attention much) Watson should essentially always win a buzzer battle. That particular piece of technology (responding immediately to a visual signal) is pretty old and should trigger at microsecond levels.

Out of curiosity, did they eliminate video clues from the game or is Watson up to those as well?

Ghoulish Delight · 02-16-2011, 01:46 PM

There were no video clues in the first go around.

02-14-2011, 11:37 AM	#2
Ghoulish Delight I Floop the Pig Join Date: Jan 2005 Location: Alternative Swankstyle Posts: 19,348	Read this about it the other day, written by Bob Harris, 8-time Jeopardy winner and participant in Jeopardy Masters Tournament. I think it's a good summation of what the event "means", if anything. __________________ 'He who receives an idea from me, receives instruction himself without lessening mine; as he who lights his taper at mine, receives light without darkening me.' -TJ
	Submit to Quotes

02-14-2011, 09:38 PM	#3
Ghoulish Delight I Floop the Pig Join Date: Jan 2005 Location: Alternative Swankstyle Posts: 19,348	So here's something I've been trying to figure out, but haven't seen anything that explains it. How does Watson understand what the categories mean. Especially the punny type categories. Does it have some definition of the kinds of puns and clue structures that Jeopardy has? Does it try to figure it out from the title? Or does it just ignore it and go purely from question content. Judging from what we saw tonight, I'm guessing option C. Compare Watson's performance in the "APB" category to the decades category. It kicked butt in the APB category since, while the clue structure was weird, it was chock full of keywords and had very straight forward answers. Just plug in the keywords, voila, answer. Whereas the decade category, just plugging in the keywords that are in the clue would perhaps bring up a bunch of dates, but wouldn't lead directly to answering in the form of a decade (and indeed, on a few of them, the Watson display showed it wasn't even considering a decade as an answer). So, if that's the case, I'm more impressed that it's really trying to figure out the KINDS of answers that it should be looking for, rather than just being fed the category definitions. Nifty! And they never show enough detail on the server racks on these things. What kind of storage are they using? What about networking. I presume the servers are clustered, and using 1GbE, but is there a separate storage network. Fibre channel? iSCSI? Infiniband? (hahahaha.........okay, I'm the only one who thinks that's funny) __________________ 'He who receives an idea from me, receives instruction himself without lessening mine; as he who lights his taper at mine, receives light without darkening me.' -TJ
	Submit to Quotes

02-14-2011, 10:29 PM	#4
lashbear SwishBuckling Bear Join Date: Jan 2005 Location: In Isolation :) Posts: 6,597	Infiniband!!! Bwah hahahahaha ! Infiniband? __________________ *I Heart* my Husband - I can't think of anyone I'd rather be in isolation with.**
	Submit to Quotes

02-15-2011, 07:35 AM	#6
Moonliner 8/30/14 - Disneyland -10k or Bust. Join Date: Jan 2005 Posts: 9,022	WTF! The final jeopardy question was leaked, I'm thinking the fix is in. Spoiler: Commonly know as a CAPTCHA what two phrases are used here to test computer users? __________________ - Taking it one step at a time.
	Submit to Quotes

02-15-2011, 03:38 PM	#7
Gn2Dlnd Parmmadore Jim Join Date: Jan 2005 Location: Casita del Queso Posts: 3,810	Sorry, your question must be phrased in the form of an answer. __________________ Does anyone still wear a hat?
	Submit to Quotes

02-16-2011, 01:21 PM	#8
Cadaverous Pallor ohhhh baby Join Date: Jan 2005 Location: Parental Bliss Posts: 12,364	This has been so interesting! Watson is killing them. I think the buzzer thing just wasn't calibrated well to allow them to get in. Watching what Watson gets wrong is so illuminating. Seems the quiz writers put in some extra effort on the Final Jeopardy question and succeeded in flummoxing Watson. Even so, his interestingly particular betting ability kept him out of trouble. The alternate answers chart on the screen including confidence in answers makes this even more fun. It cracks me up to see the computer scientists in the audience applauding their baby. __________________ The second star to the right shines in the night for you
	Submit to Quotes

02-16-2011, 01:37 PM	#9
Alex . Join Date: Feb 2005 Posts: 13,354	Well, unless they built in an uncertainty factor (and I haven't been paying attention much) Watson should essentially always win a buzzer battle. That particular piece of technology (responding immediately to a visual signal) is pretty old and should trigger at microsecond levels. Out of curiosity, did they eliminate video clues from the game or is Watson up to those as well?
	Submit to Quotes

02-16-2011, 01:46 PM	#10
Ghoulish Delight I Floop the Pig Join Date: Jan 2005 Location: Alternative Swankstyle Posts: 19,348	There were no video clues in the first go around. __________________ 'He who receives an idea from me, receives instruction himself without lessening mine; as he who lights his taper at mine, receives light without darkening me.' -TJ
	Submit to Quotes