Are images shown in Google Re-Captcha labelled by Humans?

Nowadays at many websites (which have an integration with Google’s Human Verification Engine), users are shown some set of images. And users are asked to mark some images which contain ice-cream. If the answers are correct, then the user is human. If the answers are wrong, then the user is a bot -> verified not human.

My thoughts on above scenario is the following:

The verification process assumes that only humans could clearly identify images containing ice-cream. Let us also assume that these testing images are collected from some database reachable by Google. So, these images are already labelled with correct labels so that verification process could be done.


So, my question is: Are these labelling done by humans?

  • If yes, then we are using a mechanism to identify humans based on classification done by another human. It is almost like making a statement: Only a human could identify another human.
  • If not, then these labelling is done by some image-labelling algorithm or in other words, a bot/algorithm/machine. Basically a bot is used to identify whether a user is a human or a bot. It is almost like making a statement: A bot could identify another bot.


If 2nd case is correct, following are more questions:-

  • Is the verification process valid? We are using a bot to identify a human ( Image labelling done by a bot) To what level of accuracy the images are being labelled is the next question, as currently we have research going on within context of Deep Learning.
  • We are in a situation where we are having lots of research (with increasing accuracy ) in learning mechanisms which try to learn how humans do. In this situation, giving some set of images to a bot and asking to mark would not be a difficult task either. So, should human verification process


Further thoughts:

  • We are kind of going down the path of creating robots, intelligent cars which will become more pervasive in life of humans compared to before. As before, these were restricted to industries, research area etc. We might as well reach a scenario where artificial intelligent machines would have to verify another artificial intelligent machine. Basically, what is that capability which is present only with artificial intelligent machines? So that this will be considered as verification process, as we assume now that images can be correctly recognized by humans alone.
  • Probably, ask them to calculate some largest prime number and actually feed it as the input and this could be validated.
  • This could also inspire us to ask questions like whether there are things which humans will not be able to recognize using the senses provided to a human? There are things beyond human sensory perception, which will be a baseline taken by artificial intelligent machine so that this becomes a verification mechanism for the artificial intelligent machines.


Originally published at on January 26, 2016.