Dec 21, 2019
Deep Learning and an Information Theory of Aging
By sheer serendipity, I stumbled upon David Sinclair (@davidasinclair) information theory of aging. Sinclair has spent his career in the pursuit of human longevity (i.e. seeking to cure the disease known as aging). In his recent book (“LifeSpan”), he writes about an intriguing conceptual framework as to why all biological system tends towards aging.
I am going to discuss how new discoveries about the nature of deep learning can provide an improved understanding of Sinclair’s model of aging. What I mean by this is that deep learning has given us new insights (of a computational nature) on how complex learning systems work. Can we take these new discoveries and apply them to this new theory of aging to gain greater insights? To be clear, I’m not applying deep learning to solving aging, although that indeed is a promising endeavor. I am taking a conceptual model of how deep learning works and applying this model to the understanding of aging.
So let us begin. Sinclair’s model of aging is as follows. A stem cell differentiates into a specialized cell in the body. Both the stem cell and the differentiated cell are regulated by the epigenomic information that wraps the DNA. The wrapping influences the expression of a limited amount of DNA Sinclair conjectures that aging is a result of the deterioration of a differentiated cell’s epigenomic information over time.
A cell is able to continually repair itself, but repair cycles will inevitably accumulate errors. These errors are reflected in the specialized cell unable to perform its original function. It is as if the cell has forgotten how to correctly function. In information terms, it is as if it has lost information.
David Sinclair proposes that to slow aging, one has to exercise these repair mechanisms. He notes that bacteria (and possibly all other kinds of cells) operate in at least two modes. In one mode, in an environment of rich resources, a cell tends to favor growing itself. In an alternative context where resources are scarce, a cell hunkers down and activates processes that preserve and repairs itself. The mechanisms of cells to function in these two modes is likely very ancient and therefore its control is likely in common with all biological systems. The duality of rest and activity is pervasive in biological function. Sinclair recommends that it is important to activate this preservation and repair mechanism. This is known as hormesis. This is the biological phenomenon that low dose exposure to harmful toxins can have beneficial effects.
The phenomenon of hormesis applies also to the less ancient immune system. One can characterize the immune system as a kind of liquid cognitive system. That is an intelligence that has dynamic connectivity with its components. An ant colony is also an example of a liquid cognitive system. Some would call this a liquid brain. Allergies are an example of when the immune system overreacts to the presence of non-toxic substances. One way to prevent allergies is to take micro-doses of the non-toxic molecules. The idea is to have the immune system to recognize these substances and to learn how to ignore them. I, for example, consume local bee honey so as to reduce the effects of my pollen allergies. I suspect the pervasiveness of peanut allergies in industrial countries is a consequence of the antiseptic nature of the environment. Advanced countries are too clean and thus reduces the diversity of exposure to our bodies.
When we consider the body as being a learning system, then negative perturbations expose the body’s cells to learn how to compensate for this perturbation. The cells that do get slightly damaged in these processes, learn how to repair themselves. In the absence of these ‘teachable moments’, a cell eventually forgets its skillset and eventually becomes less capable of making effective repairs. It’s like a kind of memory that requires reinforcement in spaced intervals so that it can keep itself from forgetting. Unlike the immune system, the cell aging mechanism is more ancient biological technology.
Sinclair recommends exercise, caloric restriction, exposing oneself to extreme cold and heat. The conjecture here is that one exercises the molecular repair mechanism so as to keep it efficient and thus robust. It’s the same reasoning why we have fire drills in buildings with lots of occupants. Sinclair further practices the consumption of certain molecules to enhance the activation of the ‘longevity pathways’ that triggers this cellular behavior. These include the daily consumption of NMN, resveratrol (in the morning) and metformin (at night).
There are molecules that plants and fungi produce that have very ancient effects on our functioning. Psychedelic drugs as an example trigger a molecular change in the working of our neurons to effect hallucination. In similar ways, chemicals that come from plants in harsh environments (ex: resveratrol and plants with color) perhaps trigger sympathetic cellular behavior as if the body is in a harsh environment. It is ironic that to live longer in this modern world, we have to make our bodies believe that we are living in a harsh environment. With the increase in convenience and thus the loss of hardship, we now have to know how to conveniently simulate hardship!
Sinclair likens the old epigenome as a scratched surface of a digital video disk. That is, parts of the disk become unreadable and thus its interpretation becomes flawed. So as the epigenome repairs itself, it inadvertently makes errors and these errors reflect itself in the expression (or lack of) of the genes.
In an ambitious experiment to test the hypothesis of the information deterioration of the epigenome, Sinclair applied thee of four Yamanaka factors to slightly perturb the epigenome. The Yamanaka factors (Oct3/4, Sox2, Klf4, c-Myc) turns a differentiated cell back into a pluripotent stem cell. Sinclair applied just O, S and K to perturb the epigenomic state of a cell. To his surprise, this perturbation moved the cells back to a younger version of the cell. Sinclair’s experiment was applied to a damaged mouse eye, and it was shown that damage was repaired to a state of a younger eye.
How do you know that a cell is a younger version of its current state? Apparently the methylation levels of DNA can be used as a measure of the age of a cell. This is known as the Hovarth epigenetic clock. It is likely there are many more markers to identifying age, furthermore, it appears that machine learning methods are used to disentangle this information.
It’s important to note that the application of the Yamanaka factors perturbs the cell in the direction that leads to a pluripotent stem cell. This perturbation, however, may be gradual enough that the original developmental instructions (with the presence of certain molecules) can transform the cell back into the original differentiated state of a young cell. Other regulatory molecules must be present for this regeneration to happen. Thus the discovery is that there are regulatory molecules that can move the state of the epigenome in either direction of aging. David Sinclair’s “information theory of aging” is a convincing theory as to why we age. We age because our cells forget the original intentionality that is encoded in the epigenome.
It’s intuitive as to why an organism eventually stops growing. Uncontrolled growth can be detrimental to the sustainability of an individual. Therefore, it is part of biological systems to eventually shut off the mechanism of the fountain of youth. Aoxolotl’s are known to remain in a state of neotony. But despite being able to regrow back organs, apparently, they don’t live forever (i.e. 15 years). Although an Aoxolotl has discovered how to remain young, it can’t avoid the biological clock. The mechanism for regeneration appears to be orthogonal to the lifespan clock.
Geoffrey West’s book “Scale” ponders the question of why we age. He doesn’t quite get to a model but discovers another universality in that a mammal’s average lifespan. A mammal’s lifespan is constant relative to the number of beats of its heart. Mice that have higher heart rates live much shorter lives than humans. Larger animals like elephants and whales must have slower heart rates. The universality (or power law) to the physical scaling constraints. Biological systems scale in size in a manner that is reminiscent of the branching of trees and this accounts for the universality.
The larger the animal the slower its heartbeats. It takes longer for blood to flow through a larger animal. The slower an animal’s heartbeat, the longer it can live. Alternatively, there is a fixed supply of heartbeats that is available to keep an animal alive. It reminds me of the number of battery cycles in a rechargeable battery before it can’t charge well anymore. What makes batteries degrade over time? Well, I thought we knew why, but this article (Sept 2018) says:
“Despite batteries being so ubiquitous, people don’t know how they work,” Chueh says. “It’s kind of a miracle batteries work at all.”
We don’t even know why batteries age!
But let’s get back to this abstract notion of forgetting and its relationship to complex learning systems and how information is encoded.
The nature of how DNA encodes information is still a very complex problem. We had originally imagined that we could find sections of DNA that uncontroversially identifies the instruction sequences and activates the generation of traits. The issue is that this conjecture is now in question with what is known as the ‘Omnigenic’ model.
Complex learning systems have this ‘uncertainty principle’ where the encoding incorporates both robust and random features. The oddity here is that one cannot remove the random encoding features without a detrimental effect. This uncertainty principle is also what we find in artificial deep learning systems. That is why disentanglement seems to be hopelessly difficult to achieve (i.e. training towards robust features). This characteristic of dispersing information across all the neurons leads to a more robust information system. This phenomenon may be related to what we see in how DNA encodes traits. The Omnigenic model proposes that this information is also dispersed across the entire DNA. For certain traits, DNA can be encoded over thousands of locations. This encoding of DNA may exhibit a kind of holographic encoding that we also find in entangled deep learning models.
So what new insights can we find if we take this Omnigenic model of DNA information encoding and apply it to Sinclair’s model of epigenomic information? Sinclair’s interpretation is that it is like a scratched DVD. If you’ve ever watched the contents of a scratched DVD, the scratches have a catastrophic effect on sections of the original video. Holographic encoding is more robust to errors. When one takes a hologram and cuts it in half, one can still retrieve the original information albeit in a lower resolution. But the errors are not so catastrophic as to make the information unusable. The point is here is that the dispersion of information across space leads to robust information retrieval.
The benefit of digitization (found in both DNA and DVDs) is that it permits accurate duplication of information. This is orthogonal to the value of information dispersal methods. Information dispersal leads to a robust interpretation of information in the transmission of error in the original encoding.
To illustrate this better, let me employ a generative model, specifically a stylistic GAN to illustrate the mechanics of the epigenome. The stylistic GAN has an intriguing structure in that it learns an encoding layer (see: the blocks labeled ‘A’) that it employs to constrain the generation of images.
One can think of the epigenome in this manner. That is, evolution learns a set of gene expressions that are encoded as the epigenome. This epigenome subsequently controls the generation of proteins in a manner analogous to the StylisticGAN. The effect of a perturbed epigenome is that the styles of the generative model diverge from the original.
An interesting property of deep learning generative models like GANs is that they are also adept at filling in the blanks. Here’s one example of photo reconstruction:
I suspect that the generative models that read the epigenome are similarly adept at reconstructing models despite errors in the original encoding.
What we have here is the digital information that is robust for copying. We also have a kind of dispersed holographic information that is robust to encoding errors. There exist different kinds of information that have different robustness properties. What is indeed compelling with these recent discoveries is that we are developing a new theory of information emergence that might explain not only intelligence but surprisingly also reveal insights on the nature of aging!
Further Reading
https://www.scientificamerican.com/article/aging-is-reversible-at-least-in-human-cells-and-live-mice
