Man vs App on Jeopardy tonight.

[Ghoulish Delight]

There were no video clues in the first go around.

[Alex]

But all Watson has to do is react to an electrical signal telling him when the buzzers are unlocked, which if it has already decided to answer should be an instant reflex.

For the humans there is a light that goes on to tell when the buzzers are unlocked. That unlocking is done by an assistant producer to tries to time it to Trebek finishing the question. So if the player waits until the light goes on then they're at an obvious huge disadvantage in terms of reflex.

But good players don't do that, they try to get a sense of the timing and guess when the light will come on so that they have started the press before it is actually possible to buzz in. If they time correctly it is theoretically possible they could slip into the window of whatever lag Watson has but that has to be an incredibly small opportunity. And the humans have a disadvantage that Watson doesn't, if the human guesses slightly wrong and buzzes in a microsecond early then they are locked out of buzzing in for a significant fraction of a second (that's why sometimes you see someone mashing the buzzer even when nobody else is trying).

So I don't see how, on those questions where Watson has decided it will answer it can consistently be beat to buzzing in.

[Moonliner]

Originally Posted by Alex

So I don't see how, on those questions where Watson has decided it will answer it can consistently be beat to buzzing in.

Experience is showing exactly that. If Watson buzzes in he wins a disproportionate percentage of the time. You would have to penalize Watson with a delay equal to the amount of time it takes the average human to react to a stimulus in order to even things out.

Still from another point of view, the question being asked by the Watson project is can a machine answer questions as well and as fast as the best humans. In that respect I think the answer so far is yes.

[Alex]

Sure, but to the extent it is also a test of whether a machine can win at Jeopardy it has an unfair advantage due to the buzzer mechanism.

And we're not actually learning if Watson can answer more Jeopardy questions correctly than Ken Jennings. It could be that Jennings knew 58 answers to Watson's 52 but it doesn't matter if Watson gets to answer all of the 50 they have in common.

But yes, it is certainly a great demonstration of the technology, I just quibble as to what exactly it is showing.

[Kevy Baby]

Originally Posted by Alex

... I just quibble as to what exactly it is showing.

Personally, I think it should be taken as nothing more than entertainment and a unique twist on a popular show.

[Alex]

From the Jeopardy perspective, sure (though I don't expect I'd be that entertained, for the reasons mentioned, they already knew the computer could answer the questions or it wouldn't be on the show, once that's the case most suspense is gone). But from the computer science perspective I think they are important distinctions.

[Cadaverous Pallor]

Originally Posted by Kevy Baby

Personally, I think it should be taken as nothing more than entertainment and a unique twist on a popular show.

Originally Posted by Alex

But from the computer science perspective I think they are important distinctions.

Agreed. I don't know if you're watching, Kevy, but they spent a lot of time talking about how this only one thing they can do with this technology and they hope to use it in all kinds of areas (the big example they used was health care). They're not there to demo the button pressing tech but the language parsing and decision making abilities of the machine. I was a bit disappointed in the button thing but the humans made a bit of a comeback in the last show, which was interesting. One wonders if the three of them played for days or weeks what the outcome would be.

[Kevy Baby]

We TiVod the episodes (I was out of town for a couple of days) and we just watched the first episode last night.

I was surprised to learn that Watson gets the questions as a text file: I would have thought speech recognition has evolved enough to use.

But then I started wondering - how was Watson getting the trigger to know when to push the button? But at least he has to push the same button as the players (albeit, electro-mechanically driven).

And yes, I am changing my view just a little on the show - it is a little more educational than I expected. It can be amusing to see some of the responses that Watson has been coming up with.

[Alex]

On the trigger. When the production assistant activates the buzzers after Trebek is done reading the question this turns on a light that the human players see, for Watson it sent a digital signal.

As mentioned that gives Watson an advantage since it is capable of essentially instanteous reflexes and can't (as human players risk when trying to time the activation) buzz in early causing a lockout. The other advantage is that by getting a text file Watson was probably often well on its way to an answer before the human players had even had a chance to begin comprehending what it was asking.

Like I said, it is possible that the humans knew more answers than Watson, but that is almost irrelevant if on any question that Watson also knows he is almost guaranteed winning at the buzzer.

[Kevy Baby]

So, as i see it, the two flaws in this game are:

1. Speech recognition should have been used (or, the text feed should have been streamed at the same pace as Alex's talking)
2. Warson should have to visually sense the illumination of a light (as humans must) instead of a direct link electrical signal