Tinkering with Artificial Intelligence
On the Possibility of a Post-Scientific Technology
I was very interested to see the recent paper Winner’s Curse? On Pace, Progress, and Empirical Rigor by D. Sculley, Jasper Snoek, Alex Wiltschko, and Ali Rahimi. Here is the abstract:
“The field of ML is distinguished both by rapid innovation and rapid dissemination of results. While the pace of progress has been extraordinary by any measure, in this paper we explore potential issues that we believe to be arising as a result. In particular, we observe that the rate of empirical advancement may not have been matched by consistent increase in the level of empirical rigor across the field as a whole. This short position paper highlights examples where progress has actually been slowed as a result, offers thoughts on incentive structures currently at play, and gives suggestions as seeds for discussions on productive change.”
An article in Science magazine, AI researchers allege that machine learning is alchemy by Matthew Hutson, quotes paper co-author Rahimi as saying, “There’s an anguish in the field. Many of us feel like we’re operating on an alien technology.” And then goes on to quote François Chollet as saying, “People gravitate around cargo-cult practices,” that involve, “folklore and magic spells.”
The “Winner’s Curse?” paper is seeking to address this problem by advocating research methods in machine learning that will better exemplify empirical rigor, but what if the problem is deeper than a rush to publish results and the understandable motive to be the first to announce a major innovation? What if science is not yet at the point at which it can take apart machine learning and artificial intelligence, but the engineers and technicians are on the verge of producing a reasonable facsimile, even if they do not understand (if they are successful) exactly what they have done or how they have done it?
It is entirely possible that strong general artificial intelligence (GAI) or machine consciousness will come about not as a result of a scientific research program, but rather as the result of tinkering with computers and their software. The distinction between knowing and not knowing how a result is obtained may sound like hair-splitting, but it is an important distinction: in the one case, we know what we are doing, in the other case, we do not. The importance of the latter possibility of agency without understanding cannot be overestimated, though it is, at present, merely a possibility and nothing more. How we come to artificially produce some phenomenon will bear upon what use we are able to make of GAI and who will be able to exploit this post-scientific technology.
In the past, prior to the scientific revolution, almost all practical innovations, i.e., inventions and engineering solutions to practical problems, were the result of tinkering. With the advent of science in its modern form, problems could be approached systematically, and a solution found if a solution existed. The solution was a result of deep knowledge of the problem. Science takes problems apart, analyzing them into their constituent elements, and then understands on this basis exactly what is going on. With tinkering, on the contrary, it is not deep knowledge, but a combination of practical know-how, good instincts, and a certain amount of luck that produces a solution. The solution arrived at might be possible only for an individual who possesses the practical know-how and the good instincts, which, unlike knowledge, cannot be taught. As with charisma, either you have it or you don’t.
The importance not only of GAI or machine consciousness would consist not only being able to produce these phenomena artificially, but just as much how these phenomena are produced artificially. Since the scientific revolution, tinkering has been in retreat, to the point that people who tinker in their garages without deep scientific knowledge of their efforts, are regarded today exclusively as cranks. We have become so familiar with the scientific approach to problems, and we expect so much from this approach, that we almost instinctively reject the possibility that this paradigm could come to an end. We can scarcely credit the epistemic opacity of being able to do something without being able to understand or explain how it is that we do something. We might not personally be able to give the explanation, but we have no doubt that there is someone who can give the explanation.
I have developed the Wittgensteinain theme elsewhere that, in science, nothing is hidden. That is to say, given an adequate scientific framework, any group of scientists with sufficient resources can come to the same result. One can think of this as a corollary of reproducibility. But if GAI or machine consciousness does not come about as a result of science, but rather as a result of tinkering, the principle that nothing is hidden would fail. One research team might hit upon the same practical solution to the problem as another research team, but they also might utterly fail to hit upon that solution or any other solution. Or they might hit on an alternative solution to the problem that overlaps with the solution they were trying to reproduce, but which possesses its own unique possibilities and limitations.
A new technology, and a new economy built on this new technology, could conceivably be constructed on the basis of 21st century tinkering, and this new economy would be open to participation only by those who possess the secret — for that is what we must call it, if it is not the result of scientific research — of GAI or machine consciousness. A new age of tinkering, having learned the lessons of scientific engineering but not confined exclusively to these lessons, could push our civilization in a new and unprecedented direction. That is no small thing, and, I think, it is even more interesting (if only from a theoretical point of view) than GAI or machine consciousness being the result of a scientific research program.
Technicians and engineers could, under such a technological and economic regime, forge ahead without science and scientists, because the phenomena at the heart of this regime — GAI or machine consciousness — would be their work and not the work of science. The key to further advances would be found in further tinkering rather than in further science. There would be some scientific disciplines that would be useful to the AI tinkerers — the more concrete disciplines that identify certain cognitive functions with certain brain structures — but in terms of fundamental science, basic science, knowledge of machine consciousness would be just as distant as knowledge of biological consciousness. Science would be in the dark while the modern tinkerers would continue to innovate.
We are closer to this scenario than we realize. We have no scientific understanding of consciousness whatsoever. We have some scientific knowledge of the processes of cognition and how they are related to brain structures, but that does not tell us what consciousness is. At the same time, by our own human existence proof, we know that consciousness is possible. Moreover, we can see pretty clearly that the ability to engineer consciousness for industry is potentially an economic game-changer. This is a powerful mixture of motivation and rational conjecture — powerful enough that we cannot simply dismiss it.
While GAI and artificial consciousness strike me as the most important area of technology in which a post-scientific technology could emerge from scientifically-informed tinkering, this isn’t the only possibility. Another technology that has eluded us so far is fusion power that returns more energy than is consumed by producing the fusion reaction (i.e., better than breakeven fusion). It is possible that someone tinkering with fusion technologies might be able to make the breakthrough to better than breakeven fusion, though I think this is much less likely than in the case of machine consciousness, because our scientific understanding of atomic physics is fairly sophisticated, while our scientific understanding of consciousness is close to nil.
If our existing levels of knowledge (or lack of knowledge) are correlated with the likelihood of understanding a technology scientifically, in contradistinction to arriving at a technology through tinkering, then the most likely areas for a post-scientific technology will correspond to the greatest deficits in our knowledge. Areas of knowledge that touch on human consciousness are obvious candidates. For example, the so-called “God helmet” (also known as the “Koren helmet” built by Stanley Koren and Michael Persinger) has been claimed to produce religious experiences; it has also been dismissed as a placebo effect. However, we can fairly say that the realm of religious experiences are poorly understood by science, so that a technology like the Koren helmet could be considered an example of post-scientific technology (if it does what it claims to do).
If we take a step back from the specific issue of GAI and machine consciousness, or even from fusion and the Koren helmet, and look at the difference between science and tinkering at a larger scale, we face an uncomfortable meta-problem: we don’t actually understand how or why science works. The bitter truth we must swallow is that scientific progress has been, at least in part, a matter of tinkering with science itself. The scientific method has been practiced and interpreted in different ways by different researchers, and some have even claimed that there is no such thing as the scientific method.
Another way to frame this problem is that there is no science of science. Or rather, there is a study of science, but the study of science is not itself a science, but rather it is philosophy of science. And philosophy of science cannot even agree on something as fundamental as the philosophical logic that underlies (or ought to underlie) scientific practice: some contend that science is inductive, while others argue that science is deductive. As a consequence of fundamental divergence in the conception of scientific method, if we want to improve scientific method, we have no scientific results to which we can refer. We have to tinker with science, although we don’t tinker blindly with science; we tinker with the philosophical presuppositions of science which we understand on the basis of a philosophical conception of science (whether explicitly acknowledged or passed over in silence).
We should not be surprised by this, because science itself is an outgrowth of philosophy. The scientific revolution occurred within western civilization because the peculiar history of philosophy in western civilization took a different path than philosophy in other civilizations. Arguably, philosophy was even more diverse in other regional civilizations — India, for example, has an especially prolific philosophical tradition — but it wasn’t the sheer volume of philosophical speculation that resulted in the emergence of methodological naturalism, but the nature and quality of the speculation.
While the other civilizations of the ancient world produced moral philosophies during the Axial Age, the Greeks produced natural philosophy. Western philosophy turned aside from the tradition of natural philosophy many times over in its history, but the problems of natural philosophy insistently returned, because the founding presuppositions of Greek philosophy had the problems of natural philosophy built into the conceptual framework. These problems could be avoided, but not permanently excluded.
While we rightly focus on the actual science of the early scientific revolution in the work of Copernicus, Galileo, Kepler, and others, this work did not appear ex nihilio. Many late scholastic philosophers like Jean Buridan and William of Conches were working on problems in natural philosophy, though their work took the form of commentaries on Aristotle, as was the scholastic tradition. The early scientists of the scientific revolution were more often than not well-versed in late medieval scholastic philosophy, and saw themselves as working on the same problems, albeit by a different method.
Once modern science took off, it was busy harvesting the low-hanging fruit of empirical science. At some point, the low-hanging fruit of science will be mostly exhausted, and in order to dig more deeply we will have to return to the philosophical roots of science. There are some hints that this is occurring now, but there is still a lot of hostility and misunderstanding between philosophers and scientists.
This historical situation is somewhat parallel to the social and economic problems posed by industrialization. Once industrialization became a force in its own right, transforming entire societies, industrialization has pushed forward to harvest the low-hanging fruits of economic development, and those involved in the process have reaped the economic rewards of this. But eventually, and indeed in the advanced industrialized world now, industrialization yields diminishing returns, and in order to further pursue this development we will have to return to the cluster of ideas examined in the earliest stages of the industrial revolution, thinking through these difficult problems on first principles.
It would be a great historical irony if the way forward for future industrialization were to emerge from a post-scientific technology (like GAI emergent from technological tinkering), given that the greatest economic gains of industrialization derived from the systematic application of science to problems of industry. Arguably, the earliest innovations of the industrial revolution — Watt’s steam engine and early blast furnaces — were the result of tinkering, but the most dramatic gains of the industrial revolution were from the application of science. It was only after steam engines began to make their influence felt that Sadi Carnot wrote his theoretical treatise on ideal steam engines, Réflexions sur la puissance motrice du feu et sur les machines propres à développer cette puissance, but once science was brought to bear on the problems of steam engines, things really took off.