{"id":591136,"date":"2025-04-12T13:00:00","date_gmt":"2025-04-12T11:00:00","guid":{"rendered":"https:\/\/mybroadband.co.za\/news\/?p=591136"},"modified":"2025-04-12T10:28:31","modified_gmt":"2025-04-12T08:28:31","slug":"truth-about-chatgpt-passing-the-turing-test","status":"publish","type":"post","link":"https:\/\/mybroadband.co.za\/news\/ai\/591136-truth-about-chatgpt-passing-the-turing-test.html","title":{"rendered":"Truth about ChatGPT passing the Turing test"},"content":{"rendered":"\n

There have been several headlines<\/a> over the past week about an AI chatbot officially passing<\/a> the Turing test.<\/p>\n\n\n\n

These news reports<\/a> are based on a recent preprint study<\/a> by two researchers at the University of California San Diego in which four large language models (LLMs) were put through the Turing test. One model \u2013 OpenAI\u2019s GPT-4.5 \u2013 was deemed indistinguishable from a human more than 70% of the time.<\/p>\n\n\n\n

The Turing test has been popularised<\/a> as the ultimate indicator of machine intelligence. However, there is disagreement about the validity of this test. In fact, it has a contentious history which calls into question how effective it really is at measuring machine intelligence.<\/p>\n\n\n\n

So what does this mean for the significance of this new study?<\/p>\n\n\n\n

The preprint study by cognitive scientists Cameron Jones and Benjamin Bergen was published in March, but has not yet been peer-reviewed. It tested 4 LLMs: ELIZA<\/a>, GPT-4o<\/a>, LLaMa-3.1-405B<\/a>, and GPT-4.5<\/a>.<\/p>\n\n\n\n

The tests consisted of participants completing eight rounds of conversations in which they acted as an interrogator exchanging text messages with two witnesses simultaneously. One witness was a human and the other an LLM. Some 284 participants were randomly assigned to be either the interrogator or the witness.<\/p>\n\n\n\n

Participants were required to interact with both witnesses simultaneously across a split screen for five minutes, with the test interface mimicking that of a conventional messaging interface. Following this interaction, they decided which witness was a human, and which was an AI chatbot.<\/p>\n\n\n\n

Participants judged GPT-4.5 to be the human 73% of the time, and LLaMa-3.1-405B to be the human 56% of the time. The other two models (ELIZA and GPT-4o) only fooled participants 23% and 21% of the time respectively.<\/p>\n\n\n\n

The first iteration of the Turing test was presented by English mathematician and computer scientist Alan Turing in a 1948 paper titled \u201cIntelligent Machinery<\/a>\u201d. It was originally proposed as an experiment involving three people playing chess with a theoretical machine referred to as a paper machine, two being players and one being an operator.<\/p>\n\n\n\n

In the 1950 publication \u201cComputing Machinery and Intelligence<\/a>\u201d, Turing reintroduced the experiment as the \u201cimitation game\u201d and claimed it was a means of determining a machine\u2019s ability to exhibit intelligent behaviour equivalent to a human. It involved three participants: Participant A was a woman, participant B a man and participant C either gender.<\/p>\n\n\n\n

Through a series of questions, participant C is required to determine whether \u201cX is A and Y is B\u201d or \u201cX is B and Y is A\u201d, with X and Y representing the two genders.<\/p>\n\n\n\n

A proposition is then raised: \u201cWhat will happen when a machine takes the part of A in this game? Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman?\u201d<\/p>\n\n\n\n

These questions were intended to replace the ambiguous question, \u201cCan machines think?\u201d. Turing claimed this question was ambiguous<\/a> because it required an understanding of the terms \u201cmachine\u201d and \u201cthink\u201d, of which \u201cnormal\u201d uses of the words would render a response to the question inadequate.<\/p>\n\n\n\n

Over the years, this experiment was popularised as the Turing test. While the subject matter varied, the test remained a deliberation on whether \u201cX is A and Y is B\u201d or \u201cX is B and Y is A\u201d.<\/p>\n\n\n\n

\"Statue
Statue of Alan Turing at Bletchley Park, where he helped crack the Enigma Code used by the Nazis during World War II. Photographer: Lenscap Photography \/ Shutterstock.com<\/figcaption><\/figure>\n\n\n\n

Why is it contentious?<\/h2>\n\n\n\n

While popularised as a means of testing machine intelligence, the Turing test is not unanimously accepted as an accurate means to do so. In fact, the test is frequently challenged.<\/p>\n\n\n\n

There are four main objections to the Turing test<\/a>:<\/p>\n\n\n\n

    \n
  1. Behaviour vs thinking<\/strong>. Some researchers argue the ability to \u201cpass\u201d the test is a matter of behaviour, not intelligence. Therefore it would not be contradictory to say a machine can pass the imitation game, but cannot think.<\/li>\n\n\n\n
  2. Brains are not machines<\/strong>. Turing makes assertions the brain is a machine, claiming it can be explained in purely mechanical terms. Many academics refute this claim and question the validity of the test on this basis.<\/li>\n\n\n\n
  3. Internal operations<\/strong>. As computers are not humans, their process for reaching a conclusion may not be comparable to a person\u2019s, making the test inadequate because a direct comparison cannot work.<\/li>\n\n\n\n
  4. Scope of the test<\/strong>. Some researchers believe only testing one behaviour is not enough to determine intelligence.<\/li>\n<\/ol>\n\n\n\n

    So is an LLM as smart as a human?<\/h2>\n\n\n\n

    While the preprint article claims GPT-4.5 passed the Turing test, it also states:<\/p>\n\n\n\n

    \n

    the Turing test is a measure of substitutability: whether a system can stand-in for a real person without [\u2026] noticing the difference.<\/p>\n<\/blockquote>\n\n\n\n

    This implies the researchers do not support the idea of the Turing test being a legitimate indication of human intelligence. Rather, it is an indication of the imitation of human intelligence \u2013 an ode to the origins of the test.<\/p>\n\n\n\n

    It is also worth noting that the conditions of the study were not without issue. For example, a five minute testing window is relatively short.<\/p>\n\n\n\n

    In addition, each of the LLMs was prompted to adopt a particular persona, but it\u2019s unclear what the details and impact of the \u201cpersonas\u201d were on the test.<\/p>\n\n\n\n

    For now it is safe to say GPT-4.5 is not as intelligent as humans \u2013 although it may do a reasonable job of convincing some people otherwise.<\/p>\n\n\n\n

    \n\n\n\n

    Zena Assaad<\/a>, Senior Lecturer, School of Engineering, Australian National University<\/a><\/em><\/p>\n\n\n\n

    This article is republished from The Conversation<\/a> under a Creative Commons license. Read the original article<\/a>.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

    ChatGPT just passed the Turing test, but that doesn’t mean AI is now as smart as\u00a0humans.<\/p>\n","protected":false},"author":340972,"featured_media":591137,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_sma_x_autopost_status":"idle","_sma_x_autopost_error":"","_sma_x_post_id":"","_sma_facebook_post_id":"","_sma_instagram_post_id":"","_sma_threads_post_id":"","_sma_x_attempts":0,"footnotes":""},"categories":[92837],"tags":[23004,83065,99552,84945,84789,85683,73842,45266,28232],"class_list":["post-591136","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","tag-alan-turing","tag-chatgpt","tag-eliza","tag-generative-pre-trained-transformer-4-gpt-4","tag-large-language-model-meta-ai-llama","tag-large-language-models","tag-meta-platforms","tag-openai","tag-turing-test"],"_links":{"self":[{"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/posts\/591136"}],"collection":[{"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/users\/340972"}],"replies":[{"embeddable":true,"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/comments?post=591136"}],"version-history":[{"count":1,"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/posts\/591136\/revisions"}],"predecessor-version":[{"id":591140,"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/posts\/591136\/revisions\/591140"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/media\/591137"}],"wp:attachment":[{"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/media?parent=591136"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/categories?post=591136"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mybroadband.co.za\/news\/wp-json\/wp\/v2\/tags?post=591136"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}