Developing Human-Level Artificial Specific Intelligences
An Artificial General Intelligence (AGI) is an AI agent that works across a wide domain of tasks and can perform higher cognitive functions that would benefit humanity in innumerable ways. Potential benefits include the ability to prioritize tasks, work empathically, and learn new skills when necessity dictates. Such AGIs have eluded researchers and engineers for years; however, we do have several forms of artificial intelligence that are highly adept at performing a specific task.
The term “specific” is used in psychology and intelligence studies to describe intelligence that is confined to a specific domain or modality. We use the term Artificial Specific Intelligence (ASI) to describe artificial (computer) intelligences that perform well on a constrained domain of tasks. ASIs have been developed to play games (Silver et al., 2017), perform reinforcement learning and program induction tasks (Lake et al., 2015), and even annotate cells involved in neurodegenerative disease (Signaevsky et al., 2019). Most of these systems are designed to have little or no input from human experts or supervisors.
Yet, for the most part, the AI community has failed to develop and implement truly practical applications for ASI. While the AI community at large may be content writing fancy equations with arg maxes and arg mins, it’s time we realized that no one gets paid to draw circles around bicycles in cityscapes — this is not a job function. We need more specific and useful applications for our ASI. Our AI agents can and should be picking fruits and performing other kinds of human-level work.
AIs that perform a specific job function are clearly of higher value than many present use cases. As a result, there is an urgent need for standard operating procedures when it comes to developing and deploying these kinds of ASIs.
In the following, I provide a conceptual and procedural framework for creating more useful ASIs. Lacking a standards body, anecdotes are provided from established projects. Additionally, it should be noted that this is just a preliminary outline of one approach for developing human-level ASIs. Yet, it is a framework that has proven to be successful in previous work.
The Interview
The first procedural step that members of the AI community need to integrate into their workstream is networking with individuals outside their area of expertise. In fact, one could say that our current ASIs’ deficiencies stem from computer scientists’ failures to communicate with experts in other fields— very few would argue that identifying and masking images on a background is an important, outstanding area of research in need of investigation in fields outside of computer science.
Networking and communication is a critical part of developing ASIs that perform specific job functions because each industry has its own distinctive set of procedures, quality control steps, and other characteristics that are indicative of quality and performance in the field. Machine learning scientists and researchers might call these features an objective function, or an inverse loss function. It’s what the worker is trying to do.
Current AI developers lack a method for developing their ASIs that adhere to the standards of the job function itself. Consequently, central to the development of any ASI is the interview. The interview really is just what it sounds like: You ask the person performing the job what they do, what considerations they make, what problems they encounter, how often they encounter them, what they do when they encounter these problems, and so on.
The interview helps AI developers understand what types of tasks are involved, how much knowledge is required, what type of knowledge is required, and what type of specific intelligence is required. Interviews can be as short as 20 minutes, or they may require a detailed multi-day site visit.
Two case studies myself and my colleagues have conducted using this method involve medical claims processors and neuropathology MD/PhD physician-scientists. We found that it was easy to automate highly repetitive search and classification tasks using a deep neural network, and we discovered that F1 scores in the .85-.92 range were achievable with 2,200–3,000 examples. These examples are small patches of neurofibrillary tangles that are taken from post-autopsy Alzheimer’s disease brains, and this type of task is regularly performed by pathologists in the course of their hospital service. All healthcare organizations, from the smallest clinics to the largest hospital systems, desperately need the ability to parallelize and distribute this kind of repetitive knowledge work across their organizations.
The Data Curator
The vast amount of training data that an AI agent uses to interpret complex inputs, including domain-specific expert knowledge, must be curated. The curation can be done either by an individual with particular domain knowledge or by an individual with more of an archivist or librarian skill set. This curation often includes organizing data as well as selecting data and combining datasets.
Unfortunately, many datasets are poorly curated and cheaply selected, and the abilities of the curator and the quality of the dataset and will both be reflected in the quality of the system. Consequently, methodologies must be carefully scrutinized and curators carefully selected before any work can be completed. One can think of the curator as the expert teacher of the AI — if you have a bad teacher, you will learn the wrong things.
Addressing edge cases
In virtually every scenario, there will be edge cases or task-specific problems. Human-level ASIs need to be able to distinguish and cope with edge cases and unusual inputs or behaviors. Without this type of error handling, the programs will crash or harmful downstream consequences, like factory production problems or car crashes, will occur with great regularity.
People can make value judgments and executive decisions based on their knowledge. Supervisors, for example, generally have the experience to resolve the problems of the workers they are supervising, but they sometimes need to ask others for advice. In the same way, developing a vetted scenario to handle and appropriately deal with edge cases can increase the autonomy of an ASI worker in dramatic ways. Escalating to a human supervisor is as valid for an ASI as it is for a human knowledge worker.
Areas for technical improvement
When it comes to standardizing the development of ASI, many of the outstanding issues have to do with culture, social science, management and leadership, and even worker psychology — they are not mathematical equations or computer programs. To this end, by integrating astute information management, data pre-processing, and adept curation a great variety of valuable tasks and job functions could be performed by an ASI.
From a technical standpoint, some of the most pressing areas that are in need of improvement are data models, ontologies, concept understanding, goal and task-oriented work, and the integration of supervisors with subprograms. Collecting high-quality data is an ever-present and often expensive process. Many issues persist in the integration of imaging data, trained backbone weights, text and NLP data, and time series data. GPU resources are often constrained, particularly when using large microscopy or other high-resolution datasets.
Conclusions and Future directions
Future use cases for ASIs that integrate larger knowledge bases beyond computer vision datasets include:
- Task-oriented work that involves multiple steps (program learning, program induction)
- Goal-directed work that involves balancing multiple priorities (learning with a supervisor)
- Creative work that involves design synthesis (parametric design, architecture)
When it comes to industry-specific future use cases, some applications include:
- Business and commerce: portfolio management, crop yield estimates, econometric calculations (country GDP), smart construction, smart buildings, fashion and customization, and design.
- Scientific: research related to Alzheimer’s, Parkinson’s, synthetic biology, cancer, neurodegenerative disease, aging, and precision medicine.
As a final point, it should be noted that much of the hype about AI is just that — hype. Though some computer scientists say that computers will take over, in truth, many tasks can be handed off to machines, but few jobs can be fully automated or performed by an ASI.
We will increasingly find that this phase of technological development is incremental, augmenting human abilities rather than replacing them. And with validated and tested standard operating procedures for developing ASI systems that perform specific job functions, the AI community will be able to reduce needless and tiresome overhead, improving efficiency and freeing human workers to focus on higher-level tasks that can’t be completed by ASIs.
References
Lake, B.M., Salakhutdinov, R., and Tenenbaum, J.B. (2015). Human-level concept learning through probabilistic program induction. Science 350, 1332–1338.
Signaevsky, M., Prastawa, M., Farrell, K., Tabish, N., Baldwin, E., Han, N., Iida, M.A., Koll, J., Bryce, C., Purohit, D., et al. (2019). Artificial intelligence in neuropathology: deep learning-based assessment of tauopathy. Lab. Invest.
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017). Mastering the game of Go without human knowledge. Nature 550, 354–359.
