Some notes on a null hypothesis Turing Test test

This comes from a quick chat with @psychemedia on diagnostic software that detects when a test is being run. So why not have a human version — diagnostic test for being human:

A Turing Test for human intelligence

(PS I havent searched the literature for this and it’s probably been done to death elsewhere…)

Behind this is a reasonably interesting question — has anyone actually tested the other side of the Turing Test? Specifically to confirm the Null Hypothesis before the tests can be made?

To put it another way:

How do we know that most humans would pass the test?

You see, we need to know the answer to be satisfied that humans are capable of carrying out the test itself. Ironically, this is a nice Gödellian Incompleteness / Turing Incomutable type of problem — how on earth can

One way to do this is to demonstrate a sort of Null Hypothesis (technically it wouldn’t be a pure Null Hypothesis — it would be part of the method setup). But we need to test a full control group of humans to use as the benchmark for actually passing the Test itself!

In other words, what percentage of people have to fail before we can say what a non-artificial intelligence is. 1%? 5%? 49% …

49% may not be so ridiculous, by the way. Most people think they are above average in many things so why shouldn’t this apply when it comes to being human too?

But what if, when statistically tested, it came out as 51%? What if most people assessed most other people as artificially intelligent?

What if humanity failed a Tutring Test?


  1. It could be used as a test for the simulation hypothesis
  2. It might (at a HUGE push) be extended to theories of human entropy —
  3. Perhaps it says more about human intelligence than artificial intelligence
  4. It would provide a radically new form of e-assessment and quality control in education!
  5. If used as a basis for electroal reform, we would wake up to a very different kind of polity.

Of course, the ethical issues here would have to be explored…