Déjà vu

Our family moved from Hong Kong to the United Arab Emirates in 2022, after a challenging period during the pandemic, punctuated by homeschooling and mandatory three weeks quarantine after every travel. This feels like a déjà vu: our teenagers have been homeschooling for over a month and the airspace in the region closed for a week with the initial wave of strikes. Instead of looking at the daily number of Covid-related deaths per country in the world, I was monitoring the daily number of missiles and drones intercepted by the UAE air defence system. It helped to know that the loud bangs and walls vibrating were a sign of successful interceptions. Better stick to the facts than let my mind fill in the gaps…

Normal-ish

With homeschooling and working from home, our routines were disrupted. People reacted differently. Some left the country, through evacuation assisted by their embassies or companies, some decided to stay. Fortunately, after the surprise of the first days of strike, life came back to a certain normalcy: restaurants and supermarkets remained open, public transport and taxis were operating. Nevertheless, we continued receiving alerts of interception on our phones, urging us to seek shelter for a certain duration — usually 15 to 20 minutes.

Re-encounter

Obviously, the conflict in the region is testing my resilience, and expanding my capacity to deal with uncertainty. Although I am not afraid for my life — I can only imagine what it is like to be in Tehran, Beirut or Tel Aviv where missiles actually hit the targets — I am still experiencing anxiety and restlessness due to my heightened vigilance. Yet, my daily triathlon training and commitment to looking at possibilities help me to show up every morning.

The most surprising discovery nonetheless was how my conversations and interactions with people shifted. This is the focus of this article.

I had coffee with a dear Iranian neighbor who was raised in the UAE and educated in the UK, and our familiar “how are the kids? how is work?” that shaped the beginning of all of my conversations like an imposed etiquette suddenly transformed into a raw, deep heart to heart dialogue. We shared our roller-coaster feelings, worries and hopes. I was almost in tears. I no longer saw the perfectly groomed neighbor with flawless hair (whom I envy — I admit — because my hair looks like I just got out of the pool). No, I saw a fellow human being trying to make sense of the craziness. An authentic, heart-opening moment. My conversations with other school parents also had this new flavor. In times of crisis, do we no longer have the time nor need to hide behind our social persona? Each moment counts. In each encounter, my actions and words are stripped to the essentials.

Perspective

What is really humbling as we navigate through these uncertain times is hearing other people’s experiences. My Syrian hairdresser, who lived over 10 years in the UAE shared that she was not afraid nor disturbed by the missile alerts, as she feels completely safe in comparison to what she has endured in Syria. I have Lebanese friends in the UAE who are living this conflict with a heavy heart.

My experience with war so far was limited to the stories of my father who was a boy during WWII and how he supported his mother in occupied Belgium, while his father was a prisoner of war. I grew up in France, and I remembered the Iraq war in 2003 that I followed through the news. That was it.

Stay or leave?

Given the current conflict, I can now relate to people who choose to stay because of the place they call home. After only four years in the UAE, I feel this is my home, a place where I reinvented myself as a data scientist and triathlete, a place where I expanded my understanding and appreciation of the Middle Eastern culture.

There is no “right” decision. Talking to people who fled their homes to build a new life in the UAE, like my Syrian hairdresser or my Lebanese Arabic teacher who left Beirut to raise her daughters in the UAE, I understand that whatever the decision, it comes with a cost. I made that choice when I left Hong Kong during the pandemic. I still grieve the life we had there during 15 years, but in the end, the Hong Kong I left is no longer there. With this in mind, I choose to embrace my life in the UAE and continue creating new possibilities — one step at a time.

Humble beginnings, who knew I could run

I started running in 2014, while on holiday in Phuket. A new year resolution that actually stuck. I always identified myself as a yogi so this was a big deal. We lived in Hong Kong at the time and I discovered its vibrant trail running scene. My husband and I started racing as a way to explore new trails and also to push ourselves. Hello fellow type-A trail runners in the city!

The closest I got to the podium was 4th place — in 2017. I finished the 50km with 1,719m of elevation Green Power Race in Hong Kong in 5h24. What a surprise: I was alone for the last part of the race and thought I was lost. Typical with my glorious orientation skills. The idea I was leading never even crossed my mind.

Ten years later, I’m still running

I am now based in Dubai and found a Running club that trains at 5am four times a week. Some are trail runners and tri athletes, most are road runners. Our trainings are structured with cadence runs, track sessions, and hill repeats.

Last month, I ran the Berlin marathon. It was epic this year, and I finished in 3h47. My target was sub 4, but my secret target was 3h45. Alas, I did not get that BQ (Boston marathon Qualification) but what a fantastic experience.

Owning the podium

My friend and neighbor, a gorgeous athletic mom, joined us for a track session this week. While we were driving there with my husband, she mentioned a race she participated in during the weekend and how tough it was (think 35 degrees with high humidity). Then, while we were training, she disclosed, en tête-à-tête, “by the way I was on the podium”. She got 2nd place.

My competitive husband has become a real athlete. He trains daily and often ends up on podiums. He finished the Berlin marathon in 2h57. What strikes me is that he has no problem to say that he won a race or finished 3rd.

Why do we feel, as women, that we are “bragging” when we win?

When I hear my teenage daughters downplaying their successes while my son is very vocal about his achievements, I cringe. Talking about parenting in 2024… I still need more guidance.

For now, time to celebrate my ten years of running. If a podium is in the cards, I will definitely write about it.

I. Take Rest Seriously

How many times do I find myself saying “I’ll rest when I have time”, and that time never comes. Taking rest seriously means making time for rest. Waiting until we have more time, or after we completed all the tasks of the day is the recipe for not resting at all. The author calls it “deliberate rest”.

II. Focus

By creating unbroken periods of work during the day. The author suggests 4 to 5 hours of work divided into 90-minute chunks. Intensity is more important than duration. Having tight deadlines is in fact more helpful than a date far away in the future, at least this works for me so I create deadlines when they are not imposed. This demands no distractions during short and intense periods of time, and of course accountability.

III. Layer Work and Rest

By alternating periods of work and rest during the day. Resting deliberately after intense work session produces results.

Also, align your schedule to your energy levels. My energy fluctuates during the day. Mornings work best for cognitive work like tackling data science projects or creative work. After lunch, I prefer holding meetings and handling more administrative/planning type of work.

IV. Get an Early Start

I grew up hearing my dad say “the early bird catches the worm”. I don’t know if that shaped me or if I’m naturally an early riser. Being the first at the office when I worked in Paris gave me great satisfaction to get things done with no distraction, especially as I worked in an open space. It was also the perfect strategy to avoid the crowd in the metro before peak hour. In Hong Kong, my running buddy Cordula and I went off to the trails for an early run before the busy day ahead. I started the day energised and ready to embrace whatever chaos was coming.

V. Detach from Work

During rest periods, rest FULLY, as this determines the quality of rest. This is so hard but crucial. If you’re checking emails during lunch with your spouse then guess what, you’re still at work. Multitasking is overrated, yet so common. When Cordula couldn’t make it one morning to the trails, instead of plugging my airpods to listen to a podcast, I would go run alone and let my mind wander. That’s when ideas came to me, out of nowhere.

The author describes this is as “default mode network”, when our mind organises past memories and works on unresolved problems, without any conscious effort or direction. This is when scientists or creatives have breakthroughs.

“Rest is not only restorative, it’s also creative.”

VI. Detach from Devices

Analog rests and holidays are the best. Ten years ago, we enjoyed a luxurious family vacation at The Farm at San Benito, in the Philippines. There was no wifi in the rooms, only in the lobby, deliberately. At the time, this idea was outrageous to me and now I see its value. I highly recommend the book, Digital Minimalism by Cal Newport. This book opened my eyes! Funny enough I read it after my Vipassana retreat, and since then, my rapport with social media and instant notifications has never been the same.

My children hate me for this, but I take their tech away, including their smart watch, for periods of analog time at home. It’s a constant negotiation and until now they play the piano or read books during their free time. They even get bored!

VII. A Week Off Every Three Months

That’s the ideal formula. Studies suggest that recovery period hits a peak at 8 days, and that the psychological lift of a vacation lasts about 2 months. In Dubai, we are lucky to have 2.5 day long weekends. Fridays and Saturdays used to be the weekends, and since 2022, Saturdays and Sundays are. UAE aligned with the rest of the world AND added Friday afternoon as a bonus. If you can’t take a week off every three months, don’t worry. How you spend your holiday matters more than the length.

“The only bad vacation is the one you don’t take.”

VIII. Deep Play

Find an activity that gives you the same rewards as work but without the frustrations. Playing chess, painting or rock climbing are great examples that are psychologically engaging and physically challenging.

Learning how to juggle with three balls is that kind of project for me. No big deal if I don’t succeed but how rewarding (and fun) it is to juggle for a few minutes without missing a ball. I’m still working on it by the way.

IX. Physical Exercise

The most restorative rest is active. Sounds counterintuitive? Check how you feel after watching a TV show versus a work-out. In addition to the long term health benefits you get from exercising, your brain’s plasticity also expands. You perform better when you live a physically active life.

These days, my exercise routine includes an early morning yoga session, and an evening movement session outdoors, 4 to 5 days a week. That’s a significant part of my day. I work less hours with more intensity, and gain in productivity.

X. Enough Sleep

Finally, sleep does not provide only physical rest, it also enables our brains to consolidate memories and skills. It is restorative on the physical and mental levels. Most of us know the cost of poor sleep, the impact on our performance and well-being.

The National Sleep Foundation (NSF) recommends 7 to 9 hours of sleep for adults between 26 and 64 years old. Easier said than done.

I hope these 10 commandments by Dr Alex S Pang inspire you to take rest seriously and contemplate the idea of getting more done by resting more.

Electronic products have the highest waste footprint (kg/product)

In this study, 11 products were analysed: chicken, beef, an electric drill, a laptop computer, a liter of milk, a pair of trousers, a pair of leather shoes, a smart phone, training clothes (a T-shirt and a pair of shorts in polyester), carton milk packaging and a newspaper.

The production of a 3kg laptop computer generates 1200 kg and a 169g smart phone 86 kg. If we take the smart phone, fuel and electricity account for 230g of waste, and mining and beneficiation for 85kg. In the mining industry or extractive metallurgy, beneficiation is any process that improves the economic value of the ore by removing the gangue minerals, which results in a higher grade product and a waste stream (source: Wikipedia).

One kg of beef generates more waste (4 kg) than one kg of chicken meat (860 gram). One liter of milk has a relative low waste footprint (97 gram) but its waste footprint increases around 10 percent when the footprint of its packaging (9 gram) is added to it. The waste footprints of clothing (pair of trousers 25 kg, training t-shirt and shorts 17 kg) and footwear (pair of leather shoes 12 kg) also deserve the attention of consumers. A copy of a newspaper proved to have a small waste footprint (25 gram). The main sources and reasons of waste generation are described in this report.

Electronic products have the highest climate cost

In line with the waste footprint, electronic products have a higher impact on the climate. In the figure below, the climate cost of greenhouse gas emissions related to the waste generated during production of the analysed goods was calculated in Swedish Krona (SEK). 10 SEK is around 0.9 USD at publish date.

I came across this study while doing some research on datasets on Beautiful News Daily. I stumbled on Sweden Sends Almost No Trash to Landfill — which caught my eye, and by clicking the source of the dataset, Avfall Sverige, I discovered this study. I was compelled to share.

When I think of the all electronic devices we have in our household, with 3 children equipped with smart phones and laptops for school, this study will be a great conversation starter when one will ask for the newest device.

Externalities

In economics, an externality or external cost is an indirect cost or benefit to an uninvolved third party that arises as an effect of another party’s activity. Externalities can be considered as unpriced goods involved in either consumer or producer market transactions (source: Wikipedia).

In our case, buying a phone or laptop has an “invisible cost” that is challenging to quantify but that needs to be considered in addition to the price tag.

How can we, as consumers, be more aware of these “invisible waste” and most importantly the actual cost it has on our environment?

Are carbon taxes the only way?

She was born in January this year and we welcomed her in our home when she was 8 weeks old.

Puppy Blues

After having raised 3 babies, I knew about baby blues, but never heard of puppy blues. Potty training and crate training her at night was a labor of love! I forgot how sleepless nights could transform me into an upset and irritable person. It was all worth it of course, but somehow I believe newborns are easier, maybe because they have nappies and doggies don’t.

Daycare and Boarding

Once Dune had all her vaccinations, we sent her to daycare a few times a week, so she could socialise with other puppies. She is an incredibly friendly Golden Retriever — maybe a little too friendly sometimes I fear that she could just walk away with a stranger. I was planning on taking her with me to France, to see the great outdoors away from the overheated summer in the UAE. The veterinarian advised us to wait because of her young age, so Dune went to boarding for a month.

Parvo

I loved receiving daily updates with pictures and videos of Dune from the boarding center. Dune was having a great vacation, running and playing with all her furry friends. Then, one day, she was isolated because of runny stools. Nothing to worry about I was told. The next day she was out playing with her friends again as her stools were back to normal. Then 2 days later, she was isolated again for the same reason. The boarding center staff took her to the veterinarian and that’s when the news broke: Dune tested positive to Parvovirose.

First I felt guilty of leaving her at a boarding center, then upset at the inefficacy of the vaccinations. The hospital sent me pictures of her on IV fluids. I was heart broken.

Canine Parvovirose is lethal especially with puppies. It’s a highly contagious disease that causes acute gastrointestinal illness. Symptoms often include lethargy, depression, and loss or lack of appetite, followed by a sudden onset of high fever, vomiting, and diarrhoea. I’m writing this because it’s useful for all puppy owners to know.

Happy Ending

Two days of hospitalisation, hourly monitoring and agressive treatment, Dune is recovering well and now isolated at home for several weeks. We will of course, not walk her outside, as she is still contagious.

I wrote this for future dog owners to raise awareness on this dreadful disease. Dune was lucky as she was treated early. Do keep an eye on your puppy even when fully vaccinated!

My phone doesn’t even tempt me, I look at him instead. He takes my hair with one hand, and with the other, chisels like a sculptor. He is extremely focused at the task. I am fascinated. He asks me a few questions: “how long do you want your fringe?”, “where does your hair naturally part?”. He only seeks confirmation because clearly he knows better than me.

I look around the salon and while I am amused by the chatters on the holidays, the inflation, the heatwave, I don’t need the distraction. I am absorbed by my hairdresser’s movements, how he masters his craft. His concentration is palpable and reminds me of a sushi master who excels in doing just one task at the time: slicing the fish, spreading the wasabi and placing the fish on the rice.

When manipulating my hair, his touch is not too delicate, yet not aggressive. It’s rather a firm grip that puts into shape his vision of the hair cut. My hair is delighted and dances in his hands.

Before I leave, he shares one piece of news: he will move to another town in September. The rents have skyrocketed and his landlord sold his flat. He adds, many “locals” are leaving because there are no affordable housing left. This fleeting moment, my hair’s encounter with his hands will stay memorable.

Brownies and Juniors, camping on an island

My daughters began their Girl Scout journeys as Daisies. Little did I know I was going to co-lead two troops (one for each daughter) for 4 years. My children grew up in the urban breathtaking densely populated city, Hong Kong. One of the city’s hidden gems is the incredible outdoors, the lush “mountains” and its tropical jungle, its white sandy beaches (one of my favorites is Millionaire’s beach), and its numerous surrounding islands that give you a taste of Old Hong Kong. After having lived there half of my life: 6 years when I was an expat child and 15 years as an adult — Hong Kong feels like home to me.

After a period of homeschooling, and restricted group gatherings in public areas, we organised an overnight camp for the two troops. It was October, and the weather was perfect. Many of the Juniors had a younger sister in the Brownie troop, and they all attended the same primary school. Sisterhood. We didn’t sleep much but our feet were dirty and our eyes sparkling. I was elated. On the way back home, I received a call from family in France: my 85 year old Dad was going to have surgery in the next couple of days and there was a chance he was not going to survive.

Aurora borealis

I jumped on the first flight to France. I could not sleep. I was having 100 conversations in my head, imagining 100 scenarios. The flight was dead empty, very few traveled out of Hong Kong because of the mandatory three-week hotel quarantine when you came back. Suddenly I was brought back to my seat when I heard the incessant shutter sounds of passengers snapping pictures with their iPhone. I looked out of the window, and a luminous light green trail was floating among the stars. My first encounter with the Northern Lights.

Time is of the essence

I landed in Lyon and was greeted by my Mom. My parents divorced in the 80s and then again in the 90s. It was messy and complicated. I needed consolation after a sleepless flight. Instead I was consoling my Mom, who was panicked by the news about her ex-husband’s health condition. My heart was heavy during the whole car drive. I was not only experiencing my own pain, but hers as well, entangled with all the unfinished stories of their relationship. She dropped me at my Dad’s where I was welcomed by my step mom and two younger brothers. The other brothers were on their way to Lyon. We called the doctor to know exactly when the surgery was scheduled and to our surprise, there was no surgery scheduled. Dad was diagnosed with gallbladder cancer and given his condition, only palliative care was an option. At that moment I was relieved and the pressure of our “last conversation” was released.

Syndrome de glissement

Seeing my Dad at the hospital was a moment of mixed feelings: the joy of being reunited after 2 years of not travelling and the awareness of how much weight he had lost. His appetite was low and meal times were a struggle. We brought him his favorite dishes, home made. I couldn’t blame him for not wanting to indulge in the hospital food. One of the nurses who took care of him, whispered to me in private that perhaps his lack of appetite was due to the “syndrome de glissement”, translated by the “geriatric failure to thrive”. I never heard of this French medical term, highly debated in France. “Glissement” means slipping, and the image is of a person slipping out of life, losing interest in food and other aspects of living, like a form of depression of the elderly, who not consciously, are letting go of life. Whether or not he had this syndrome, what struck me was how meal times became a moment of confrontation: I was adamant that he had to eat if he wanted to gain strength and get out of the hospital. I felt like I was spoon-feeding a rebellious toddler. I was fighting for him to recover. A fight that was not mine. After many tears and frustration, I let go. You can’t force a toddler to eat, how could I expect to force my Dad?! My inner narrative changed and my focus was to meet him wherever he was, not trying to go anywhere or push him. Our conversations transformed and I was so much more present to my Dad. I even enjoyed just sitting next to him, in silence.

My uncertain return to Hong Kong

A month passed since I left my husband and three children in HK. I was grateful to our friends who all lived in the same neighborhood and who came to visit frequently. Our friends became our family: since nobody travelled, it was a unique time in HK when our friendships really blossomed to a new level. It was quite ironic how the two years of “lockdown” in HK were by far the best years in terms of friendships — perhaps because there was nowhere to go. My Dad’s wish was to return home, asap. I found palliative medical care, a home nurse and a physiotherapist so he could walk autonomously again. This was epic as I navigated through the flawed French system of healthcare for the elderly. It was December and I was longing to go back home too. Flights to HK were cancelled. I kept moving my quarantine hotel reservation. Even as a permanent resident, I could not board a flight to HK if I did not have a confirmed and paid booking at a hotel with a negative PCR test delivered by a laboratory from an approved list. This is why nobody travelled! Eventually, I found a flight via Amsterdam on 24 December.

Christmas Eve at an airport hotel

I was wearing a FFP2 mask at my Mom’s place to minimise my chance of contracting Covid. I was ready to board the flight in Amsterdam when after a 2 hour line at the boarding gate with agents checking everybody’s paperwork, I was refused to board: the laboratory where I had my PCR test done was not part of the approved list. I was in tears. The ground staff said their airline would be in trouble with the HK authorities if they let me in. I checked in at the Sheraton hotel in Schiphol and slept in despair. I flew back to Lyon the next day, and had Christmas lunch with my Mom and step dad. Plan B was in motion: the next flight was a week later and I was going to nail down the requirement list. I also enjoyed a few extra peaceful days with Dad, a lovely bonus!

21 days in Quarantine

I landed in the ghost airport of HK on 31 December 2021 and was waiting for the result of my 2nd PCR test, mandatory for all passengers who arrived. The clock struck midnight when I was in the queue for the hotel shuttle bus. I was relieved I made it this far, now I was 21 days away from squeezing my children and husband. The experience of quarantine was surprisingly exquisite! I was lucky the hotel was outstanding and almost everyday I received delicious treats and flowers from my dear friends. I kept my mind and body healthy with meditation and a skipping rope. I spoke to my Dad on the phone everyday : he too was making progress with walking. Little wins that warmed my heart.

My last two months in Home Kong

Freedom day: the day I left the quarantine hotel. The transition to my “normal” life was bittersweet, somehow I longed to be in my quiet “bubble” again. I continued to call my Dad every other day. Our conversations were simple, and revolved more and more around what was going on in the present (his coffee was too warm, this part of his body aching), and I listened. A couple of months later, Carrie Lam, the chief Executive of HK, announced the school closure and that was it. Enough was enough. The shadow of months of homeschooling we endured the previous year was lurking and there was no way we would experience that again. We decided that my husband would fly to France with the children within the next few days and I would join them 2 weeks later giving me sufficient time to take care of the anticipated move. Moving out from HK to Dubai was a plan created over the past year and we were supposed to stay until the end of June.

The precipitated move

This was my first “big move”. During our 15 years in HK, we moved 4 times in different neighborhoods as our family grew. This time it was different: we were moving out of the country and we could not host a farewell party. It was mid March and I had lunch with my childhood best friend in a new restaurant. She asked me how I was dealing with the situation of my father’s palliative care. I said confidently “if this had happened 10 years ago, I would have been devastated. Right now, I feel I am in a place where I can handle it.” Those words.

Unsaid goodbyes

The next day, I was getting ready for my daily morning jog, when my phone rang at 6am. My brother in France called to say that Dad had passed away a few minutes ago and that he did all he could to revive him. I sat down on the stairs, letting the news sink in. I jogged to the waterfront and there I paused a moment, looking at the sea. I called my Mom and my brother R. Sharing the news was already painful but hearing their reaction shattered my heart. I was adamant in maintaining my schedule for the day: coffee with a friend and lunch at home with 2 very close friends. A part of me was honoring my word, and a part of me was thinking if I were to cancel my day’s appointments, I would be crying in bed, alone. It helped to have a “normal” day, and to share this moment with close friends. A week later, I flew to France knowing that I was leaving Hong Kong for good.

One year later

Grieving my Dad has expanded my experience of life and death, and dare I say, humanity. At moments I felt lost and engaged in reckless activities, almost died, and injured myself. Looking back, I numbed my pain with high adrenaline pursuits and wanted to feel alive.

I am writing this post from our mountain cabin in France where I will stay for the summer. No paragliding — although I look up sometimes to the sky with envy! You can read the beginning of my Data Science journey here. So far, our new chapter in Dubai is promising: we welcomed Dune, a Golden Retriever puppy in our new home and, everyday is filled with discovery and new encounters. The protective walls are slowly melting away and I can say with 95% confidence (pun intended!) that I am in a newfound place. There is light at the end of the tunnel and so much more beyond!

Decision Trees

A decision tree is an algorithm used in Supervised Learning (read my post on Understanding Decision Trees). One of the limitations of using decision trees as a tool for predictive learning is overfitting: they work great on training data but are not flexible on unseen data. In other words, the error of misclassification of a new sample is high. To overcome this, we use the Random Forest algorithm, which combines the simplicity of decision trees with flexibility.

What is a Forest?

Instead of one decision tree trying to solve a problem, we have many decision trees working together as a team (the forest!). Each decision tree has its own strength and weakness. Each tree makes a prediction and the prediction with the most votes (in a classification task) or the average of the predictions (in a regression task) is the final prediction.

When we use a random forest, we start by creating many decision trees, each with a different subset of the data and a different subset of the features.

Building a Forest

1. Create a “bootstrapped” dataset (by randomly selecting samples from an existing dataset, with replacement)
2. Create a decision tree using the bootstrapped dataset and use a random subset of variables (features) at each split
3. Repeat from Step 1

Each tree is generated for each “bootstrapped” dataset. The ensemble of decision trees compose the Forest.

Using a Forest

Let’s say we want to predict whether a person will finish a marathon or not, based on their age, gender, previous race results, training frequency, average pace, weekly mileage, etc. Each marathon organiser builds their own decision tree, based on the dataset of runners they already have. So, for example, one decision tree might focus on the runner’s weekly mileage and previous race results, while another might focus on their average pace and age. The final prediction is then the majority vote of the predictions of all the decision trees in that forest.

Now the model is trained on the existing datasets of each marathon organiser. Let’s say we have 50 decision trees in the Forest. A new runner registers for a marathon, and we want to predict whether he/she will finish the marathon. We enter his/her data (age, gender, previous race results, training frequency, average pace, weekly mileage, etc.) in the model and each decision tree will give its prediction based on the patterns it learned from the existing datasets with this new data. The result is 45 decision trees out of the 50 predicted that the runner will finish the marathon, so the final prediction is the runner will finish the marathon.

Bagging (Bootstrap Aggregating)

Bagging combines multiple models trained on different bootstrap samples. Random forest is a bagging algorithm where the base models are decision trees.

A random forest also uses a subset of randomly selected independent variables (features) at each node’s branching step (rather than considering all features), which increases diversity among the trees.

Randomness

Random forest adds additional randomness to the model while growing the trees. Instead of searching for the most important feature while splitting a node, it searches for the best feature among a random subset of features. This results in a wide diversity that generally results in a better model.

Random selection of features at each split further increases the diversity in the model and hence the independence of the weak decision trees from one another. A higher number of trees, increases the diversity and robustness. This would make them overlap more and if we use the best feature (from all the features) at each split then more and more trees would be similar and less random.

So what is a good number of features to consider at the split?

The number of features that can be searched at each split point (m) must be specified as a parameter to the algorithm (max_features). You can try different values.

• For classification a good default is: m = sqrt(p)
• For regression a good default is: m = p/3

And how many trees should compose the forest?

A larger number of trees can improve the performance of the model, but also increases the computational cost. One way to find the optimal number of trees (n_estimators) can be done through a grid search over a range of values, and select the value that gives the best performance on the testing set. The process of finding the maximum number of features and number of trees on the test data is called tuning.

Time for a hike — let’s explore the real physical forest around me!

Structure

A decision tree is an algorithm used in Supervised Learning (read a simple explanation in my post on Supervised and Unsupervised Learning). It is like a flow chart with decision nodes that split into child nodes.

Decision trees are simple to understand and interpret. Each node is a test on a feature (eg. is it morning?) and each branch is the outcome of the test (eg. yes or no). The top node is called the root node. The bottom nodes that not split anymore are the leaves. The various paths from the root to leaf are the decision rules.

Application

Decision trees can be used to solve either Regression and Classification problems. For example, if you want to predict the weight of a person or any other continuous numeric quantity, then this is a regression problem. On the other hand, if you want to classify whether this animal is a bird or a reptile, or any other category, then this a classification problem.

How to build a decision tree

You pick a feature (root node), you split the data based on that feature so that the outcome is binary (no data points belong to both sides of the split) and you define a new decision rule. You repeat the process until each leaf node is pure or homogeneous (all the data points in a leaf node belong to the same class). This process is called recursive partitioning.

How to optimise the split?

In order to find the best split in decision trees, we can use scoring metrics such as Entropy and Gini Impurity which help up rank the features.

Entropy and Information Gain

Entropy is a measure of uncertainty in a random variable. If X is a random variable with the probability mass function p(X), then the entropy of X is:

E represents the expected value. We use the logarithm to the base 2.

In decision trees, we are interested in the entropy of the target variable for a given split. This is called conditional entropy: H(T/a), entropy of T given a.

The Information Gain (IG) is the difference between the entropy of the parent node and the conditional entropy of the child node for a given split:

The lower the entropy of the child nodes, the larger the information gain is. In other words, information gain is the reduction in entropy. We split the nodes at the most informative features by maximising the information gain at the split.

Gini Impurity

Another metric to determine how well a decision tree is split is the Gini impurity which tells us what is the probability of misclassifying or mispredicting a datapoint. The lower the Gini impurity, the better the split. In other words, the lower the likelihood of misclassification. If we have a dataset of classes C, and P( i) is the probability of picking a data point of class i, then the Gini Impurity is:

Advantages

Using decision trees compared to other Supervised learning algorithms has many advantages:

• simple to understand and explain (interpretability)
• able to handle both numerical and categorical data
• performs well with large datasets
• requires little data preparation
• naturally de-emphasises non relevant features (no need for PCA)

Limitations

However as my teacher says “there is no free lunch”, here are some of the limitations when using decision trees:

• a slight change in the training data can result in a big change in the tree, and therefore predictions (non robust)
• the algorithm doesn’t guarantee obtaining the globally optimal decision tree (greedy algorithm)
• overfitting: they work well with training data but are not flexible when classifying new samples (unseen data), in other words, they don’t generalise as we increase the complexity (depth of the tree, number of features)

In conclusion, it is easy to understand why decision trees are among the most popular supervised learning algorithms given its simplicity and interpretability. Its limitations however, will lead us to my next post: Wandering in Random Forests.

“When all the covariance matrices are the same for all classes, the boundary is linear in X, and so the QDA algorithm becomes LDA.”

Once I am able to explain the above to a non technical person, I will write a post on that! For now, I’ll put that aside as I am still figuring this out.

What is a Covariance matrix?

First of all, let’s define covariance. Covariance means how much one variable increases or decreases when another variable increases or decreases. For example, let’s take 2 random variables, X and Y. Let’s say that X is Jack playing football, and Y is Jill eating popcorn. The covariance measures how these two variables (X, Y) vary together: Jack playing football (X) and Jill’s popcorn consumption (Y). Covariance is a number that tells us how much they vary together.

The mathematical formula is:

The covariance matrix is simply a way of representing the covariances in a square matrix. In our example, we have a 2 dimensional covariance matrix where the diagonal values from top left to bottom right show the variances (X varies against itself) and the other values are the covariances (X varies against Y, and Y varies against X):

What is a Correlation?

Correlation, on the other hand, measures how strong the relationship is between the two variables, in other words, how much one variable is related to the other. In our example, if Jack plays a lot of football, does that mean Jill will always eat a lot of popcorn? Is the relationship between the two weak or strong? Correlation is a number that tells you how strong the relationship is between the two variables.

The mathematical formula for the correlation coefficient is:

So, in a nutshell, covariance measures how much two variables vary together, while correlation measures how strong the relationship is between the two variables. Please check the excellent and more detailed explanation on the Careerfoundry blog post on this topic.