What is the Turing test?

Turing's thought experiment

In 1950, British mathematician Alan Turing asked: "Can a machine think?" In his paper 'Computing Machinery and Intelligence' he described a test he called the imitation game, later known as the Turing test.

The idea is simple: a human judge conducts a text conversation with an unknown conversation partner. If the judge cannot determine whether they are speaking with a human or a machine, the machine has passed the test.

Illustration created with Canva AI

Has AI already passed the Turing test?

In 2014, a chatbot called "Eugene Goostman" claimed to have passed the test. However, this was controversial: the judges knew they might be speaking with a 13-year-old Ukrainian boy, which lowered the bar.

Modern LLMs like GPT-4 and Claude convince most people in informal conversations that they are talking to a human. Formally, however, the test has never been passed without controversy.

Criticism of the Turing test

• The philosophical argument — John Searle's 'Chinese Room': a system can give the right answers without 'understanding' anything
• The deception argument — passing the test requires convincing, not understanding
• Not comprehensive — the test only measures linguistic performance, not reasoning, creativity, or emotion
• Outdated — designed when computers could barely calculate, let alone converse

More modern alternatives

Researchers have proposed several alternatives:

• ARC (Abstraction and Reasoning Corpus) — tests abstract reasoning that humans find easy but machines find difficult
• BIG-bench — a broad benchmark of hundreds of tasks
• MMLU — academic knowledge testing across 57 disciplines

Turing's legacy

The Turing test is outdated as a measure, but remains culturally and philosophically relevant. It made us think about what intelligence is, what it means to 'understand', and where the boundary lies between human and machine. Questions that are more relevant than ever.