Study exposes major flaw in classic artificial intelligence test

A serious problem in the Turing test for computer intelligence is exposed in a study published in the Journal of Experimental and Theoretical Artificial Intelligence.

If a machine were to 'take the Fifth Amendment' – that is, exercise the right to remain silent throughout the test – it could, potentially, pass the test and thus be regarded as a thinking entity, authors Kevin Warwick and Huma Shah of Coventry University argue. However, if this is the case, any silent entity could pass the test, even if it were clearly incapable of thought.

The test, devised in 1950 by pioneering computer scientist Alan Turing, assesses a machine's ability to exhibit intelligent behaviour indistinguishable from that of a human. Also known as the 'imitation game', it requires a human judge to converse with two hidden entities, a human and a machine, and then determine which is which.

Warwick and Shah's study looks at transcripts of a number of conversations from actual Turing tests in which the hidden machine remained silent. In each case, the human judge was unable to say for certain whether they were interacting with a person or a machine.

Thus, a machine could potentially pass the Turing test simply by remaining silent. The judge would be unable to determine whether the silent entity was a human choosing not to answer the questions, a smart machine that had decided not to reply, or a machine experiencing technical problems that prevented it from answering (as was actually the case in the transcripts studied).

Kevin Warwick said: "This begs the question, what exactly does it mean to pass the Turing test? Turing introduced his imitation game as a replacement for the question 'Can machines think?' and the end conclusion of this is that if an entity passes the test then we have to regard it as a thinking entity."

"However, if an entity can pass the test by remaining silent, this cannot be seen as an indication it is a thinking entity, otherwise objects such as stones or rocks, which clearly do not think, could pass the test. Therefore, we must conclude that 'taking the Fifth' fleshes out a serious flaw in the Turing test."