The chatbot that 'passed the Turing test'

On June 8, 2014, the University of Reading issued a press release headlined “Turing Test success marks milestone in computing history.” It announced that “the 65 year-old iconic Turing Test was passed for the very first time by computer programme Eugene Goostman” at an event held at the Royal Society in London. The program, developed in Saint Petersburg, “simulates a 13 year old boy.” The bar, per the release, was that “if a computer is mistaken for a human more than 30% of the time during a series of five minute keyboard conversations it passes the test”; Eugene “managed to convince 33% of the human judges (30 judges took part).”

The announcement traveled around the world as “a computer finally passed the Turing test.” The careful reading is less dramatic. Alan Turing’s 1950 paper “Computing Machinery and Intelligence” proposed the imitation game as a way to sidestep the vague question “Can machines think?” - but Turing described a sustained game of identification, not a fixed 30% threshold over five-minute chats. The 30% figure comes from a single speculative passage in his paper about what might happen by the year 2000, not a defined pass mark.

The design of Eugene mattered too. Casting the bot as a 13-year-old Ukrainian boy speaking English as a second language gave judges a built-in excuse for odd answers, gaps in knowledge, and broken grammar - exactly the failings that would otherwise expose a machine. The “pass” was engineered around the test’s weak points as much as it was achieved.

The episode is a clean example of how a milestone can be technically claimed and substantively hollow. Nothing about Eugene suggested understanding; it suggested that a short conversation, a sympathetic persona, and a literal reading of one sentence in a 1950 paper can manufacture a headline.