Rina Dechter —Deep Learning Pioneer
Today deep learning is a hot topic in the fields of artificial intelligence and machine learning, but it was a woman who back in 1986 coined the term. Meet Rina Dechter, pioneer in computational aspects of automated reasoning and knowledge representation and the first person to use the phrase deep learning in a scientific paper.
Introducing the Concept
In her article “Learning While Searching in Constraint-Satisfaction Problems” published in 1986, she introduces the concept of Deep Learning. One of the aims in her paper is to focus in constraint recording when dead-ends arise as a process of learning. By analysing and storing the reasons for the dead-ends you can then use them to lead future decisions, so that the same conflicts will not emerge again, or to backjump to the appropriate relevant state rather than to the last one in chronological time.
She called deep learning the idea of getting to know all the possible information out of a dead-end, meaning recording all the search space explored in order to improve the learning performance. Here you can see a fragment of the paper:
Deep Learning
Today the concept of deep learning is associated to computers gathering knowledge from experience and learning complicated concepts by building them out of simpler ones.
Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction.
The challenge of artificial intelligence has been solving tasks that are easy for people to perform but hard for them to describe formally, like recognizing spoken words or faces in images. The idea is that computers learn from experience and understand the world in terms of a hierarchy of concepts. In this way humans don’t need to specify the knowledge to computers because they gather it from experience. The hierarchy of concepts enables the computer to learn complicated ideas by building them out of simpler ones. The key aspect of deep learning is that these layers of features are not designed by humans, quite the opposite, they are learned from data using a general-purpose learning procedure.
Graphical Models and Constraint Processing
Going back to our pioneer, one of Rina’s major contributions has been describing that reasoning with uncertainty within machine learning can be performed with Graphical Models (graph-based representations and reasoning algorithms). This can include constraint networks (hard and soft), Bayesian networks, Markov random fields, and influence diagrams.
Graphical models provide powerful tools for solving problems in a range of application domains. For example, within the area of natural language processing (information extraction, semantic parsing and translation), computer vision (object recognition, scene analysis, segmentation and tracking), computational biology (pedigree analysis, protein folding and binding and sequence matching), networks (webpage link analysis, social networks, communications and citations) and robotics (planning and decision making).
She is also responsible for introducing the temporal constraint framework, central in any scheme for planning and scheduling, which can be seen in her book Constraint Processing (2003), and the bucket elimination framework, which unifies dynamic programming for combinatorial optimization, probabilistic reasoning and planning under uncertainty.
Rina Dechter, an ACM fellow has had contributions in the algorithmic foundations of automated reasoning with constraint-based and probabilistic information. With over 50 research papers to her name, and having worked on the editorial boards of Artificial Intelligence, Constraints, Journal of Artificial Intelligence Research (JAIR) and Logical Method in Computer Science (LMCS), makes her one of the most important researchers in AI.
Advent Calendar — Help us make it a book!
From December 1st until December 24th we plan to release one article each day, highlighting the life of one of the many women that have made today’s computing industry as amazing as it is: From early compilers to computer games, from chip design to distributed systems, we will revisit the lives of these pioneers.
Each article will come with an amazing illustration by @SebastianNavasF
If you want to see these series to become a book with expanded articles and even more illustrations by Sebastián, then subscribe to our newsletter below.
Credits
- Illustration: Sebastián Navas
References
- The Resurgence of Artificial Intelligence During 1983–2010 (https://datafloq.com/read/resurgence-of-artificial-intelligence-1983-2010/4795)
- Goodfellow, Ian, et al. Deep learning. Vol. 1. Cambridge: MIT press, 2016. (http://www.deeplearningbook.org/)
- Rina Dechter and David Cohen. Constraint processing. Morgan Kaufmann, 2003.
- Rina Dechter. Learning while searching in constraint-satisfaction problems. University of California, Computer Science Department, Cognitive Systems Laboratory, 1986.
- Rina Dechter Personal Website (https://www.ics.uci.edu/~dechter/index.html)
- Wikipedia Profile (https://en.wikipedia.org/wiki/Rina_Dechter)
