AI Scientist — A self-evolving Architecture of the Theory Machine
Can we extract the essence of brilliance from people like Albert Einstein or Alan Turing into a machine? I believe so. Together with Michal Kovacik, we propose a Theory Machine via a research paper: THEORY MACHINE - BRIDGING GENERATIVE AI CREATIVITY AND TURING MACHINE FORMALITY A PROPOSAL
Most researchers today are pursuing the holy grail of self-tuning generative AI, as it signifies a self-improving AI without human assistance, edging us closer to the technological singularity.
An effective method is generating synthetic data with a larger model, then feeding it into a smaller one, essentially simulating imagination.
Humans and other biological creatures have evolved the ability to generalize from small data sets for survival. Originally a survival tool, imagination now fuels creativity. For instance, if a caveman encountered a large predator and survived, their imagination would run wild, simulating various scenarios, training the brain for future encounters.
So, why can’t AIs generalize as well? Imagine this scenario:
A software engineer is tasked with teaching programming to someone in an empty room, devoid of computers, pens, paper, or furniture. Teaching might be possible, but it would be extremely slow and yield unreliable results.
This is akin to today’s generative AIs teaching other AIs via synthetic data. It works, but not reliably. Furthermore, teaching new and emergent abilities is very hard, if not impossible.
Now, introduce a computer into this room. The situation changes dramatically. The learner can quickly grasp programming, test new knowledge, and potentially create new, emergent knowledge.
So, what can we learn from this?
Why were Albert Einstein, Stephen Hawking, and Alan Turing so successful with their theories and considered intelligent? I believe it’s their ability to describe the world with rigorous formality, making their theories exceptionally testable. They created Total Turing Machines—well-defined rules—upon the world’s Recursive language, finding patterns in the incoming world noise to build theories.
Take Einstein’s journey to the special theory of relativity. He navigated a tumultuous landscape of conflicting theories in physics. Traditional Newtonian mechanics clashed with new evidence about light and electromagnetism, such as the failure of the Michelson-Morley experiment to detect ether. Einstein’s imaginative thought experiments, like envisioning traveling alongside a beam of light, allowed him to mentally sift through the noise, leading to his revolutionary insights on light speed and altering our understanding of time and space.
We aim to encapsulate this process in the Theory Machine:
As the Theory Machine observes the world, like Einstein did, it seeks patterns in the data, conducting thought experiments.
The incoming data can be anything: visual, textual, binary, or a combination.
Upon identifying a pattern, or a Recursive language, the machine generates a hypothesis. If we can build a Total Turing Machine (that never loops) on this hypothesis, it becomes a fully testable environment, our Proposed Turing Machine.
Testing more noise data on the Proposed Turing Machine, if we get reliable runs, the hypothesis could advance to Theory status.
Further training the Foundational Model on these results creates a feedback loop, acting as a Reward Function.
This approach emulates the scientific method’s rigorous process, similar to how Einstein’s hypotheses were empirically tested. A concrete example is the 1919 solar eclipse observation by Sir Arthur Eddington, confirming Einstein’s general relativity theory. The Theory Machine aims to distill complex real-world phenomena into a Total Turing Machine, mirroring this process of theory confirmation through empirical evidence.
The essence of the Theory Machine is to create a broad enough Reward Function, or Process Reward Model, enabling Foundational Models to train and adapt with minimal or no human intervention.
Like AlphaGo trained itself in the narrow domain of the Go game, the Theory Machine could theoretically train for anything.We don’t know where this self-evolving architecture might lead us, but it could be seen as a machine specialized for new scientific progress—an AI scientist.
The Theory Machine’s ability to develop and intuitively understand Turing Machines could lead to new, specialized, and efficient Microchip designs. For instance, for a specific use case like developing an AI data center or creating a car manufacturing factory, you could describe the use case to the Theory Machine, which could then create specialized microchips for extreme efficiency.
Moreover, the Theory Machine could generate cross-domain designs, like bio-microchips, though its potential remains to be tested.
This article was partially written in Kurt Gödel’s birthplace: Brno, Czechia.
Without Kurt Gödel, Alan Turing wouldn’t be inspired and we probably wouldn’t have computers.
