A Rational Analysis of Curiosity

Rachit Dubey, Thomas L. Griffiths
(Submitted on 11 May 2017)

We present a rational analysis of curiosity, proposing that people’s curiosity is driven by seeking stimuli that maximize their ability to make appropriate responses in the future. This perspective offers a way to unify previous theories of curiosity into a single framework. Experimental results confirm our model’s predictions, showing how the relationship between curiosity and confidence can change significantly depending on the nature of the environment.

Introduction
Curiosity, the topic of this paper, is a rather interesting one. It is related to the field of intrinsic motivation, and is recognized as a factor in a wide range of human endeavour. Curiosity is inherent to many scientific contributions, and plays an important role in the domain of creative arts as well. The authors of this paper present a rational analysis of curiosity, and propose that people’s curiosity is driven by seeking stimuli that maximizes ability to make appropriate responses in the future. The paper presents a model of curiosity and show through experimental results how the relationship between curiosity and confidence can change depending on the nature of the environment.

Existing Models of Curiosity

Curiosity based on Novelty
This theory was first published in the British Journal of Psychology, Volume 41, Issue 1–2, September 1950, pages 68–80 by British/Canadian psychologist D.E. Berlyne. This theory hypothesized that curiosity is linked with novelty, as gaining information of a novel stimuli has higher intrinsic reward. Berlyne termed this “perceptual curiosity” as a driving force that motivates an individual to seek out novel stimuli which decreases with exposure. The limitation of this theory is that it assumes an optimal solution for an individual to explore novel stimuli in all environments. In practice, there are many cases in which the environment may have a novel stimuli, but lacks the information or resources to allow an individual to study the stimuli and gain useful knowledge.

Curiosity based on Information Gap
This theory was proposed by Loewenstein in the Psychological Bulletin, Volume 116, Number 1, page 75–98, 1994. This theory hypothesis that curiosity arises when individuals has a gap in information, and thereby prompting the individual to complete the missing knowledge and resolve uncertainty. However, curiosity diminishes when one knows too little or too much about a stimuli. One limitation of this theory is it fails to take into account the landscape of an individual’s pre-existing interest when one’s informational reference point becomes elevated in a particular domain. Perhaps further investigation is needed to explain why certain people become interested in certain topics and why certain topics are almost universally interesting.

Curiosity based on Learning Progress
The guiding hypothesis behind this theory is that learning progress generates intrinsic reward. This theory hypothesizes that the brain is intrinsically motivated to pursue tasks in which one’s predictions are always improving. Thus, individuals will focus on tasks that are learnable rather than tasks that are too easy or too difficult. The limitation of this theory is again in the environment. If the environment is filled with difficult tasks, it is not clear how the curiosity of an individual will work.

Proposed Model of Curiosity
Before I go into the model, I will list the notations used for this model.

Experimental Hypotheses
The rational model of curiosity makes two empirical predictions.

Dependency Between Need Probability and Exposure

Dependency Between Need Probability and Exposure

Experimental Setup
The details of the behavioral experiment performed to test the hypotheses in the previous section can be found in the paper. The purpose of the experiments were to assess whether people’s curiosity is affected by changes in the relationship between the need probability and confidence. Therefore, the expected results were the right hand sides of figures 2 and 4, which showed an “inverted U-shape” relationship.

Experimental Results
If you did not read the details of the experimental setup, the experiment consisted of 2 rounds. The first round (aka the main round) was that participants were shown 40 questions and asked to rate their confidence to answer the question, and their curiosity towards that question.

In the first part of the second round, all questions are shown to the participants and they could choose whether or not they wished to be revealed the answer. Revealing an answer comes at a cost of a 10 second delay. The time it took to go over all the 40 questions is used as an indicator to measure curiosity (participants who are more curious will ask for answers to questions and therefore, take more time).

In the second part of the second round, participants were given a fixed amount of time to answer up to 10 questions from the previous round. The sampling consisted of two types; random sampling and sampling based on confidence. Random sampling just samples up to 10 questions uniformly, while confidence based sampling selected questions for which the participants’ confidence rating was higher.

The results display that the relationship between curiosity and confidence indeed exhibits an “inverted U-shape” behavior. This was measured using the time the participant took to go over the 40 questions in the main round and is shown in figure 5. I would comment on the nearly identical nature (at least from my perspective) of the two curves. Since all 40 questions were shown to the participants of both types of sampling, the type of sampling doesn’t really play an effect in this result. Therefore, we see an universal behavior between both groups of participants.

Another result displayed is the probability for a participant to ask for an answer to be revealed vs the participants confidence in answering that question. This is shown in figure 6. I will comment on the uniform sampling result in this case. It displays that participants are typically more inclined to ask for answers when they are less confident about a question, which is very intuitive. Therefore, as the participants confidence of a question increases, the probability of asking for a reveal decreases.

Conclusion and Final Comments
The paper presents a model of curiosity based on the agents environment, stimulus, and the drive to maximize the probability to make appropriate responses in the future. The authors also discuss briefly other ways of looking at curiosity.

The results suggest that that human curiosity is not only sensitive to the properties of the stimuli but it is also affected by the nature of the environment. If people are in an environment where need probability influences exposure, then their curiosity is highest for stimuli for which they are moderately confident about. On the other hand, if need probability and exposure are independent of each other then curiosity is highest for novel stimuli.

As a final comment, I personally think the model is extremely crude. As explained previously, by modeling curiosity as equation (1) does not take into account the difficulty of the stimuli and participants will have reached 99% confidence with just 3 exposures to that stimuli. In addition, the the statement “curiosity will be highest when the agent is moderately confident about a stimulus” made in hypothesis (i) is not correct when comparing with the confidence based sampling results in figure 6. Finally, it is somewhat
true that the uniform sampling results in in figure 6 does seem like it is monotonically decreasing (but if you look closely, that is not the case for low confidence), it is nowhere representative of their hypothesis (ii).
Therefore, I think the contribution of this paper is more of an interesting idea on how humans can try to capture the abstract, intangible idea of curiosity. It may inspire AI researchers to think of ways of which a machine will evaluate and choose the next task. However, much improvement on the model is needed to be able to put any of this to practical use in the realm of AI.

Source: https://arxiv.org/abs/1705.04351

Author: Joshua Chou | Reviewer: Qintong Wu