The Interplay Between Artificial Intelligence and Uncertainty
In this first article, we highlight how intelligence and rationality are tightly coupled with the uncertainty present in the world. We also discuss how uncertainty plays a critical role in designing beneficial general-purpose artificial intelligence (AI), as described by the work of Stuart Russel and Peter Norvig on Modern AI [1][2].
Human intelligence, both social and individual, is what has been driving advances achieved by the human civilization. Having access to even greater intelligence in the form of machine artificial intelligence (AI) can potentially lead to even further advances, and will help us solve major problems such as eliminating poverty and disease, solving open scientific and mathematical problems, and offering personal assistance targeting billions of people worldwide. This is subject of course to the finite resources of land and raw material available on earth.
Scientists differentiate between narrow AI that is designed to perform a narrow task, and that it may outperform human beings at this task, and general-purpose AI that outperforms humans at nearly every cognitive task. An example of narrow AI is the deep learning techniques that produced starting 2011 huge advances in speech recognition, visual object recognition, and machine translation. Machines now match or exceed human capabilities in these areas. On the other hand, we can imagine general-purpose AI to have access to all the knowledge and skills of our human race, with their embodiments in the real world just differing in physical capabilities depending on the application.
Narrow AI is becoming a pervasive aspect of our present life, and it is making the news headlines on a weekly even daily basis. It is hard to predict when super-intelligent general-purpose AI will arrive, but nevertheless we must plan for the possibility that machines will far exceed the human capacity for decision making in the real world. Prediction of the arrival of super-intelligent AI is difficult because as we know from other scientific fields (nuclear physics for example), scientific breakthroughs are hard to predict, and perhaps this is why there is a long history of such predictions going wrong. However, we should not deny the possibility of success when our future is at stake. We are working on developing entities that are far more powerful than humans, so we need to ensure they never have power over us.
Super-human intelligence can be the biggest event in human history and its last. To give you an example of how current narrow AI technologies, that are not particularly intelligent, can still affect billions of people around the world, consider the content selection algorithm used on social media platforms whose objective is to maximize user click-through (proportional to monetary revenue), by presenting the user with items to click on. When the user preferences are hard to predict, reinforcement learning algorithms, instead of presenting the user with items they like, will try to make the user preferences more predictable by changing the user’s preferences themselves. As we know, extreme preferences are easier to predict (think extreme left or right in politics), and so the algorithm will potentially attempt to “radicalize” the user’s mind in order to maximize its click-through reward. In this case, there is a mismatch between the human’s intended objective of increasing revenue and the AI’s realized objective of maximizing clicks by biasing people’s behavior.
Humans are intelligent to the extent that our actions (based on what we perceive) can be expected to achieve our objectives. And in the example above, we can see that we designed the machine using the same notion in the sense that its actions can be expected to achieve its objectives. The objectives are fed into the machine by a human in the form of an optimization formulation. Using this definition of intelligence, the problem in the example above occurred because the purpose put into the machine is not the purpose which we really desire. As humans, we are often uncertain about our objectives and knowledge, so if we put the wrong objective into a machine that is more intelligent than us, it will achieve the objective, but this might be catastrophic. Also, we cannot continue to rely on ironing out the major errors in an objective function by trial and error especially for machines of increasing intelligence and increasingly global impact. So, it seems necessary to remove the assumption that machines should have a definite objective as we discuss below.
We focus hereafter on understanding intelligence, human intelligence in particular, since explaining how the mind works is a step towards developing beneficial artificial intelligence. The first cornerstone of intelligence is learning, because we can use it to adapt to a range of circumstances. There is so much we still do not understand about the human brain, but one of the aspects related to learning that we are beginning to understand is the reward system in the brain. This is an internal signaling system, mediated by dopamine, that connects positive and negative stimuli to behavior. It is extremely difficult for an organism to decide what actions are most likely, in the long run, to result in successful propagation of its genes, so evolution provided us with breadcrumbs. This is closely related to the method of reinforcement learning developed in AI.
However it is important to notice that learning and evolution does not necessarily point in the same direction, since reward can be obtained by taking drugs and playing video games all day, which will reduce the likelihood that one’s genes will propagate. Similarly, our understanding of intelligence was first based on the assumption that what a person wants is fixed and known, and that rational action is one that easily and best produces the desired goal. However, this did not take uncertainty into account, where in the real world, few actions or sequences of actions are truly guaranteed to achieve the intended end. Here probability and gambling play a central role in explaining the trade-offs between the certainty of success and the cost of ensuring that degree of certainty. Furthermore, in the eighteenth century, Swiss mathematician Daniel Bernoulli, explained that bets should be evaluated according to expected utility rather than expected monetary value to reflect what is useful or beneficial to a person. Utility is distinct from monetary value and exhibits diminishing returns with respect to money. Moreover, the utility values of bets are not directly observable but are inferred from the preferences exhibited by an individual. In the middle of the twentieth century, John von Neumann and Oskar Morgenstern published an axiomatic basis for utility theory which states the following: as long as the preferences exhibited by an individual satisfy certain basic axioms that any rational agent should satisfy, then necessarily the choices made by that individual can be described as maximizing the expected value of a utility function. In short, a rational agent acts to maximize expected utility. Moreover, maximizing expected utility may not require calculating any expectations or any utilities. It is a purely external description of a rational entity. There is a lot of debate whether human beings are rational or not. It can be argued that our preferences only seem irrational because we try to compensate for the mismatch between our small brains and the complexity of the decision problem that we face all the time.
Moreover, with the presence of other humans and machines with different objectives than ours, an agent will need yet another way to make rational decisions. This is where game theory plays a big role in attempting to extend the notion of rationality to situations with multiple agents. Here, just like gambling, the trick is that every agent does not choose one action, but a randomized strategy instead. Each agent mentally tosses a suitably biased coin (depending on their strategy) just before picking an action, so they do not give away their intentions. By acting unpredictably, even if the competing agent figures out our randomized strategy, there is not much they can do about it without a crystal ball.
Based on this new notion of intelligence, AI researchers are starting to adopt the tools of probability theory and utility theory and thereby connecting AI to other fields such as statistics, control theory, economics, and operations research. This change marked the beginning of what some observers call modern AI. However, the way we build intelligent agents depends on the nature of the problem we face. We make a list of factors that can change the nature of the problem an agent is facing:
1. the nature of the environment the agent will operate in, and whether this environment is fully observable or partially observable.
2. whether the environment and actions are discrete or effectively continuous.
3. whether the environment contains other agents or not.
4. whether the outcomes of actions are predictable or unpredictable.
5. whether the rules or “physics” of the environment are known or unknown.
6. whether the environment is dynamically changing, so that the time to make decisions is tightly constrained or not.
7. the length of the horizon over which decision quality is measured according to the objective; this may be short, of intermediate duration, or very long .
Building an AI system for any of these problems requires a great deal of problem-specific engineering. On the other hand, the goal of general-purpose AI would be a method that is applicable across all problem types . The agent would learn what it needs to learn from all the available resources, ask questions when necessary, and begin formulating and executing plans that work. Again, just because such a general-purpose method does not yet exist, it does not mean we are not moving closer, and a lot of progress towards general AI results from research on narrow AI. Currently, instead of building one agent with general-purpose AI, we instead build a group of agents each addressing a different type of problem.
For each of these agents to deal with uncertainty, instead of using a goal, modern AI uses a utility function to describe the desirability of different outcomes or sequences of states. The utility is expressed as a sum of rewards for each of the states in the sequence. Therefore, the machine aims to produce behavior that maximizes its expected sum of rewards, averaged over the possible outcomes weighted by their probabilities. For this purpose, researchers have developed a variety of algorithms for decision making under uncertainty. One example of such algorithms is what are known as “dynamic programming” algorithms. These are the probabilistic cousins of lookahead search and planning. For the case when the number of states is enormous and the reward comes only at the end of the game, AI researchers have developed a method called reinforcement learning that learns from direct experience of reward signals in the environment.
In this article, we summarized how uncertainty affects intelligence and designing AI agents. In the upcoming article, we discuss the potential danger and misuse of super intelligent AI.
[1] Russel, S., Norvig, P. “Artificial Intelligence: A Modern Approach,” Pearson 2020.
[2] Russel, S. “Human Compatible — Artificial Intelligence And The Problem Of Control,” Penguin Random House 2019.
