Vipassana and Machine Learning
Vipassana is probably most well known for its 10 days meditation camps. And machine learning is the study of training computers to perform tasks without human’s explicit instructions, by observing patterns in data.
Totally different things, right?
Vipassana 10 days retreat in 20 words
To see why they are relevant, we need to have some brief ideas about what would happen in the Vipassana retreat. Here is my (opinionated) summary of the Vipassana 10 days retreat:
1. Controlled environment — noble silence, vegetarian diet, daily schedule
2. Activity — observe your sensations objectively
3. Objective — train your mind to be equanimous
Once the Day 0 started, everyone has to observe the noble silence (no contact with the outside, no chatting, no physical contacts, and even no eye contacts!) and some other rules (no killing, no lies, no stealing, no working out, no reading/writing, no improper sexual activity, etc). Other than that, one will follow a vegetarian diet and a daily schedule of 4:30 am — 9:30 pm, and meditate 10 hours (in sessions) every day until Day 10.
Does it sound like the life of a monk? Yes, it does, but there are rationales behind it, which I will elaborate later.
And what we do during the 10 hours of meditation? It can be divided into 2 phases:
- Phase 1 (Day 1 to Day 3):
Observe your respiration objectively — feel the air in and out in the nose and the area below it. Start with a larger area and gradually reduce it to a smaller area. - Phase 2 (Day 4 to Day 10):
Observe the sensation throughout your body objectively — scan your body part by part to feel the sensation, move your attention from one area to another, from head to legs and from legs to head. Again, start with observing your entire head, arms, core, and legs as a whole part, and then gradually make it more and more granular.
Other than meditation sessions, there is a 1-hour discourse around the end of every day, which consists of explanations of the concepts and the rationale behind, with a lot of stories as examples.
These are mostly what happened during the 10 days.
Vipassana’s objective and why it is useful
But what for?
The main objective of practicing Vipassana is to train your mind to be equanimous so that you can live a happier and carefree life. Equanimous in Vipassana’s sense means no craving for pleasant sensations nor aversion for unpleasant ones. In other words, do not expect pleasant sensations will stay and do not expect unpleasant sensations will go away. Just observe with a balanced and unbiased mind.
Without equanimity, one will suffer, regardless of what kind of environment he/she is in, or what kind of outcome he/she would get.
Let’s imagine you were a co-founder and the CEO of a multi-billion startup. Your mega-unicorn was very big, and then you got 2 more rounds of billion funding from a famous VC, securing a valuation of $47 billion. Then your company filed an IPO, which means you could monetarize the efforts throughout the years. Everything was so good, right? Except the IPO didn’t happen. Within one month, the valuation went from $47 billion to $10 billion, but the story didn’t end here. You were being removed from your own company, as the investors no longer trusted you to be able to lead the company. But in return, you got $1.7 billion as the condition of stepping down. How should you feel?
IF everything went well, you would be able to exit with much higher $1.7 billion, and still being the decision maker of the company. But in reality is that it didn’t well that well, and in fact quite bad. If you are unable to remain equanimous, you probably feel you lost everything — your once $47B company as well as your reputation, and then suffer from the pain and disappointment of losing everything. But if you can see things with an equanimous mind, you will realize that you still “barely” get $1.7 billion, and you are now not the one to clean up all the crap — the investors and the new CEO are.
This is actually a true story and the company is called WeWork. How did Adam Neumann feel? No one can tell exactly except Adam himself. Regardless of how he felt, what’s happened has happened. With equanimity, you are happy and carefree, and without it means pain and suffering.
Vipassana from machine learning’s perspective
Okay, but how is it related to machine learning?
Our mind is like a machine learning model. Given the input sensations, the mind goes through a lot of interpretations and convolutions, and output the feeling and reactions. Different people would have a different architecture of their “mind model”, with different weights initialization. So given the same environment settings, we may experience different sensations or interpret them differently, which yields different feelings. But we all share the same objective function — to output a feeling as equanimous as possible. Unless you have a perfect model, the output feeling is usually less equanimous than perfect equanimity.
The difference between perfect equanimity and your output feeling is conditioning. When someone blames you and you immediately feel angry, that is conditioning. When you are having some good food and you keep wanting more and even better food, that is also conditioning. Conditioning itself is a natural phenomenon so there is no right or wrong about it. But if you want to be more equanimous (so to live more happily and carefree), you may want to minimize conditioning, just like in machine learning you want to minimize the cost as much as possible.
The way to minimize the cost in Vipassana is to adjust your mind model based on the output feeling, with respect to the changes in the input sensations. If you train a machine learning model to predict credit card fraud, your model would not crave the inputs of a non-fraud case, nor would it try to avoid a fraud case. When the model fails to predict certain cases, it won’t feel sad or frustrated (though you may). What it would do is to simply adjust the weights so that the next step would have a higher chance of predicting correctly. So one thing we can learn from the model is equanimity — do not (and no need to) crave pleasant inputs nor averse to unpleasant ones, as the inputs can be anything and it will keep changing. This belief that all things are impermanent and constantly changing is called anicca in Vipassana.
The output feeling will be an input sensation feature of the next time frame, convoluting with the other sensations. So if you have an un-equanimous feeling at t=0, it will affect what the output feeling is at t=1, which in turn affects feeling at t=2, 3, … n. During my meditation sessions, I often felt frustrated when the sensations went numb or very weak. The frustration itself made me feel annoyed, and overwhelmed the other “actual” sensations. Once I gradually realized and accepted the fact that I had lost my equanimity, I was outputting a more equanimous feeling, which then went back as the input of the next step. So gradually I recovered from my un-equanimity by realizing and accepting it.
Transfer learning and Reinforcement learning
A lot of metaphors, right? Plus these metaphors apply to any kind of learning, not specific to machine learning. It is true that those characteristics can be found in other kinds of learning, but there are some characteristics that are specific to machine learning:
- Transfer learning
The mind model you trained during Vipassana meditation is actually the same thing you use in daily life. In daily life, you also experience a lot of pleasant or unpleasant sensations, just that now the sensations are mostly the stimuli from the external environment instead of directly from your body. If you can remain equanimous during Vipassana, you probably can be more equanimous in handling the ups and downs in daily life. This resembles Transfer learning in machine learning, where “where a model developed for a task is reused as the starting point for a model on a second task”. - Reinforcement learning
Technically you can practice Vipassana directly in your daily life because Vipassana is about keeping an equanimous mind, and you can practice so without the meditation process. However, practicing Vipassana via meditation offers 2 advantages over practicing via daily life:
i) lower risk — if you fail to remain equanimous during meditation, the worst outcome is you can’t experience the sensations or they are very weak and rough. But if you fail to remain equanimous in daily life, there could be real consequences like losing friends, losing money, or even your own life, depending on the reactions coupled with the un-equanimous feeling.
ii) higher speed — how many WeWork dramas could Adam Neumann experience in his life? And 1 round trip of head to legs back and forth scan in Vipassana is measured by minutes, and you can control how long you practice.
One of the reasons why AlphaGo is so strong is because it can keep training by playing against itself, interacting with the environment (i.e. the Go board and rules) and adjust the model given the outcome. Same for Vipassana and now the environment is your body.
The life of a monk — hyperparameters
Now it should make sense why they let us follow a monk’s lifestyle during the 10 days retreat — that is the hyperparameters best tuned for efficient training of the mind model. In machine learning, if you are training with a suboptimal learning rate, you may still be able to train your model, but it may converge much more slowly, or it may be trapped in some local minimum.
The same happens for the rules and diet in the 10 days retreat. You can probably train your mind to be more equanimous without following all the rules, just that it will yield worse outcomes — your mind model improves very slowly, or it got stuck.
I talked with an old student on Day 10 after the noble silence rule was lifted. He said that it was his second 10 days retreat and he had been practicing Vipassana meditation 1 hour every day for 1 year, but during the last 10 days he felt that he had made more progress than the 1 year of meditation effort in the “outside world”. That makes me think how important the hyperparameters tuning is.
Closing
It has been 1 month since I completed the retreat, and I still think it was worth it. The ideas are very simple and actually not new, and they could sound cliché if you simply read or hear about it. The WeWork story is actually the half glass of water story, and I am sure you have heard of those concepts in other books or religions as well. To me, the beauty of Vipassana is that it introduces a practical way for people to confirm they are applying those ideas to themselves and become “better”, rather than preaching the rules without practicing them.
You can read 10 posts in Distill on how to make your model better, but your model won’t be better until you train it. Or you can claim that you have read the 10 posts so your model should train better, and again your model won’t benefit from the 10 posts until you actually train it with those ideas implemented. And Vipassana has the end-to-end, carefully-tuned setup for you.
Acknowledgments
Vipassana official website: https://www.dhamma.org/en-US/index
Diagrams created via https://www.draw.io/
Some icons in the diagrams are from https://www.flaticon.com/
