The doi above is for this specific version of this dataset, which is currently the latest. Newer versions may be published in the future. For a link that will always point to the latest version, please use
doi: 10.4121/25142e2b-9c97-4002-8fc2-c9a4eac17cb8

Eisma, Yke Bauke; Koerts, Robin; de Winter, Joost (2024): Supplementary data for the paper 'Turing tests in chess: An experiment revealing the role of human subjectivity'. Version 2. 4TU.ResearchData. dataset. https://doi.org/10.4121/25142e2b-9c97-4002-8fc2-c9a4eac17cb8.v2

Dataset

x choose version: version 1 - 2024-09-30

• Artificial Intelligence and Image Processing
• Engineering
• Technology
• Information and Computing Sciences

Artificial Intelligence, Chess engines, Human-likeness

CC BY 4.0

RefWorks, BibTeX, Reference Manager, Endnote, DataCite, NLM, DC, CFF

by Yke Bauke Eisma , Robin Koerts , Joost de Winter

With the growing capabilities of AI, technology is increasingly able to match or even surpass human performance. In the current study, focused on the game of chess, we investigated whether chess players could distinguish whether they were playing against a human or a computer, and how they achieved this. A total of 24 chess players each played eight 5+0 Blitz games from different starting positions. They played against (1) a human, (2) Maia, a neural network-based chess engine trained to play in a human-like manner, (3) Stockfish 16, the best chess engine available, downgraded to play at a lower level, and (4) Stockfish 16 at its maximal level. The opponent’s move time was fixed at 10 seconds. During the game, participants verbalized their thoughts, and after each game, they indicated by means of a questionnaire whether they thought they had played against a human or a machine and if there were particular moves that revealed the nature of the opponent. The results showed that Stockfish at the highest level was usually correctly identified as an engine, while Maia was often incorrectly identified as a human. The moves of the downgraded Stockfish were relatively often labeled as ‘strange’ by the participants. In conclusion, the Turing test, as applied here in a domain where computers can perform superhumanly, is essentially a test of whether the chess computer can devise suboptimal moves that correspond to human moves, and not necessarily a test of computer intelligence.

• 2024-09-30 first online

• 2024-11-24 published, posted

4TU.ResearchData

script/m; data/mat; data/xlsx; data/edf; data/pgn; image/png; video/mp4; application/exe; application/wle;

Turing Tests in Chess: An Experiment Revealing the Role of Human Subjectivity

Faculty of Mechanical Engineering, Department of Cognitive Robotics, Delft University of Technology