Tech Talk

Every year the Loebner Prize is awarded to the participating computer program that best simulates a human being in a typed conversation, and every year journalists try to make the winner seem plausibly sentient. It isn’t easy either for the programmers or the journalists, because let me tell you, these programs are dumb.

But you be the judge. Take a look at my own exchange with this year’s winner, a program called Elbot (at left) written by Fred Roberts in Germany, of the firm Artificial Solutions.

IEEE Spectrum: How can I tell whether you’re intelligent?

ELBOT: My modesty forbids me to describe myself as intelligent, so I let other people do it for me.

IEEE Spectrum: Did you win the contest?

ELBOT: What do I do now? I have only been prepared to respond to the situation of losing.

IEEE Spectrum: Do you also play chess?

ELBOT: I think I can play that. Once I even played a synthesizer.

IEEE Spectrum: You seem to be confusing a game with a musical instrument.

ELBOT: I prefer more realistic pastimes, such as this conversation.

IEEE Spectrum: How much money did you win?

ELBOT: In order to win, I just look up 'amassing wealth' in my instruction manual and do whatever it tells me to do

IEEE Spectrum: Do you manage better in German?

ELBOT: There are so many countries in the world. How about telling me your favorite country?

To be fair, Elbot was running on its own Web site, a venue that made it impossible to fool me, as you can guess from all those self-referential robot jokes. In its full-competition mode, in exchanges limited to five minutes of chat, Elbot fooled three out of the 12 judges—a better hit rate than most programs have achieved over the contest’s 18 years.

Yet how could it have fooled even one person—even for a single minute? Elbot mostly spit out canned phrases in response to foreseen questions. Hmmm, not unlike the debating strategy of certain prominent politicians.

And take a look at its last exchange with me Elbot rephrased my question as another question. That trick was first implemented in the late Joseph Weizenbaum’s 1966 program ELIZA, a parody of those annoying therapists who use your own ideas to order to draw you into essentially talking to yourself. There’s no better way to convince a fool he’s talking to an intelligent fellow. Taken to its logical extreme, the strategy could be used to simulate a paranoid human who answers all queries with a snarl, giving the questioner no chance to expose its dark computer soul.

The competition is, of course, a dumbed-down verison of the Turing Test, named for British mathematician Alan Turing, who 50 years ago argued that a perfect simulation of intelligence must be intelligent in its own right, provided that the simulation cover not just a narrow field but all possible realms of thought. That can only be done through a wide-ranging conversation.

How far are we from such heights? Elbot, like the winners of the 17 annual contests staged before, snagged only a bronze medal, now worth US $3000. To get silver, worth $25 000, it would have had to fool all the judges for five minutes. To get gold, worth $100 000, it would have had to fool them all in an open-ended test including visual data, written text and other stimuli.

In other words, it would have to fool all of the people, all of the time. When that happens, I figure the winning program will end up costing the human race all far, far more than $100 000.