<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Allen Wang on Medium]]></title>
        <description><![CDATA[Stories by Allen Wang on Medium]]></description>
        <link>https://medium.com/@allenwang1536?source=rss-d789c5c45b7a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*620xAcz3go6ug4NwMnc66w.jpeg</url>
            <title>Stories by Allen Wang on Medium</title>
            <link>https://medium.com/@allenwang1536?source=rss-d789c5c45b7a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 15 May 2026 08:41:24 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@allenwang1536/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[Treating Alcohol Addiction With Personalized Hypnotherapy]]></title>
            <link>https://medium.com/swlh/treating-alcohol-addiction-with-personalized-hypnotherapy-e35096bb645?source=rss-d789c5c45b7a------2</link>
            <guid isPermaLink="false">https://medium.com/p/e35096bb645</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[brain-computer-interface]]></category>
            <category><![CDATA[bci]]></category>
            <category><![CDATA[psychology]]></category>
            <category><![CDATA[hypnotherapy]]></category>
            <dc:creator><![CDATA[Allen Wang]]></dc:creator>
            <pubDate>Sat, 01 May 2021 02:51:18 GMT</pubDate>
            <atom:updated>2021-05-07T17:05:39.516Z</atom:updated>
            <content:encoded><![CDATA[<h4>Introducing Lana — the product that will reshape addictive habits</h4><p>Those of us who love watching (and re-watching) the show Friends might remember how Chandler became a “strong, confident, woman” that was hypnotized to not need to smoke. He was able to do so by using a tape that would play suggestions while he was sleeping in order to change himself for the better.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*8JfqHGKf19E0lfqE.jpg" /><figcaption>Friends Season 3 Episode 18</figcaption></figure><p>While it was indeed hilarious (and one of my personal favorite episodes), I assume that most of us probably laughed it off as a sham that would never be able to happen in real life.</p><p><strong>Yet, is there any way that this might <em>actually </em>be possible?</strong></p><p>New research and studies certainly seem to suggest so. Read further to see how we utilize hypnotherapy to solve the <strong>problem of alcohol addiction.</strong></p><p><em>Before we continue, I’ve also assembled a table of contents for you to skip around to the content you’re interested in</em></p><h3>Table Of Contents</h3><h4><a href="#6e29"><strong>Introduction to Hypnotherapy</strong></a></h4><ul><li><a href="#06b2">Does it Actually Work?</a></li><li><a href="#9b4f">What Happens in a Typical Hypnotic Hypnotherapy Session?</a></li><li><a href="#c130">How Do You Measure Hypnotic Trance?</a></li></ul><h4><a href="#5581">The Problem — Alcoholic Addiction</a></h4><h4><a href="#a25b"><strong>Using Hypnotherapy to Our Advantage</strong></a></h4><ul><li><a href="#6902">Introducing Lana</a></li><li><a href="#63c4">Step 1: Getting Into a Trance</a></li><li><a href="#3084">Step 2: Implementing Suggestions</a></li><li><a href="#c0b7">Step 3: Continuation of Sleep</a></li></ul><h4><a href="#821a"><strong>Current Limitations</strong></a></h4><h4><a href="#094a">Vision For The Future</a></h4><h3>Introduction to Hypnotherapy</h3><h4>Does it Actually Work?</h4><p>Unfortunately, many have a misconception about hypnotherapy, viewing it as simply a scam magic that holds no merit. There are many myths, mainly from traditional media, that have painted the idea that hypnosis only consists of someone telling a patient to look closely as he or she swings a pocket watch back and forth.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8RfWgdxc7nZHOBvE.jpg" /></figure><p>Yet, when you dig deep into the research papers, there are <strong>many studies and experiments that have proved hypnosis to be a real phenomenon</strong>, with effects ranging from reduction of stress and anxiety to reduction of hot flashes (Here’s an excellent article that links <a href="https://hypnosistrainingacademy.com/science-behind-hypnosis-breakthrough-studies/">19 different research studies proving the benefits of hypnosis</a> if you want to read further in-depth).</p><p>It has become an AMA approved therapy, and the American Psychiatric Association has recognized it as “a powerful, effective therapeutic technique for a wide range of conditions”</p><p>In fact, Irving Kirsch, a lecturer and director of the Program in Placebo Studies at Harvard Medical School, has stated that <a href="https://time.com/5380312/is-hypnosis-real-science/">“<em>hypnosis is a well-studied and legitimate form of adjunct treatment” for conditions ranging from “obesity and pain after surgery to anxiety and stress.</em>”</a></p><p>Of course, hypnosis is not a cure-all that will work for everyone, as its effects range from person to person— but that’s the case for modern medicine as well. Thus, we can treat hypnosis as simply another alternative treatment that may work well for some, and may not work at all for others.</p><h4>What Happens in a Typical Hypnotic Hypnotherapy Session?</h4><p>During hypnosis, a hypnotherapist will induce a state of intense concentration through verbal cues and repetition. This puts one into a trance-like state which is similar to that of sleep. While you’re in this trance-like state, your therapist will make guided suggestions designed to help achieve your therapeutic goals.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*XaE6Xn5COYafyJ2X.jpg" /></figure><p>Since you are in this state of heightened concentration trance, <em>you’re more suggestible to proposals</em> that you may normally brush off in your normal state of mind. When the session is complete, your therapist will wake you from the state with another audio cue.</p><p>Though it is not exactly known why hypnotherapy works yet, most experts postulate that the change in consciousness during hypnotherapy allows for suggestions to take firmer root in your mind. However, one thing scientists know for certain — Hypnosis does work and it does produce actual effects (as shown by the studies linked earlier).</p><h4>How Do You Measure Hypnotic Trance?</h4><p><a href="https://bigthink.com/mind-brain/how-hypnosis-works-stanford-university?rebelltitem=2#rebelltitem2">According to a study by Stanford University</a>, there are three primary activities in the brain to consider when in trance:</p><p>1. A <strong>decrease in dorsal anterior cingulate activity</strong></p><ul><li>This is responsible for decision making, evaluation processes, and emotional regulation as well as physiological functions such as blood pressure and heart rate.</li></ul><p>2. An <strong>increase in the connection between the dorsolateral prefrontal cortex and the insula</strong></p><ul><li>This leads to a stronger brain-body connection; more specifically, the pain perception, social engagements, emotions, and autonomic control functions of the brain more readily impacts the executive functions such as working memory and self-control in the dorsolateral prefrontal cortex</li></ul><p><strong>3. Reduced connections between the dorsolateral prefrontal cortex and the medial prefrontal cortex</strong></p><ul><li>This heightens the disconnect between actions and awareness of actions</li></ul><p>Now, what kind of technology can pick up on these signals?</p><p>One new proposal for an intracranial EEG-based, noninvasive BCI can collect general brain activity data. This will then be analyzed by using a procedure known as <strong>network analysis</strong> to pick up on signs of these activities.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DBnBdaxCl4K-OyFu.jpg" /></figure><p>Network analysis helps us <em>determine the functional connectivity between different regions of a brain</em> during a specific cognitive task or process, and this functional connectivity can be represented through graph theory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/983/0*2RSINBvrZ4jDYZPk.png" /><figcaption>nodes = neural elements, edges = connection between neural elements</figcaption></figure><p>These edges can then be given different weights, based on how strong the connection between brain regions is. This allows us to form a “topography” that represents the pattern of weights across the entire brain.</p><p>Then, using neural input from EEG BCIs, where each electrode represents a node, we can conduct functional connectivity analysis using a method called “<strong>phase locking.”</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hBaJdMKFFv-KO5URnh7VIg.png" /></figure><p>Phase locking is the detection of where the neural signals (which come in the form of oscillations) are <em>synchronous based on frequency</em>. Between a pair of nodes, the weight is defined by the high gamma phase-locking value (70–100 Hz) for that pair.</p><p>Thus, we can use this proposed BCI analysis method to provide a portable means of measuring hypnotic trance. Now that we understand the basics and efficacy of hypnotherapy, let’s dive into the problem that it can help to solve.</p><h3>The Problem — Alcohol Addiction</h3><blockquote>“Excessive alcohol use is responsible for more than 95,000 deaths in the United States each year, or 261 deaths per day. These deaths shorten the lives of those who die by an average of almost 29 years, for a total of 2.8 million years of potential life lost.” — <a href="https://www.cdc.gov/alcohol/features/excessive-alcohol-deaths.html#:~:text=Excessive%20alcohol%20use%20is%20responsible,years%20of%20potential%20life%20lost.">CDC</a></blockquote><p>Our current system to treat alcoholism is ineffective.</p><p>The Twelve-step process that we currently use to help the 24 million Americans who suffer from substance abuse and addiction has shown to just not be enough. A heavy-handed model that is based on abstinence and Christian spirituality is not a model that works for everyone. In fact, the 12 step process usually helps only <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2746426/">25% more people than no process at all</a>.</p><p><strong>That’s not good enough.</strong></p><p>Contrary to this model, hypnotherapy is proving to be more and more effective at treating alcoholism and substance abuse (Go back to <a href="#6e29">Introduction to Hypnotherapy</a> for more details).</p><p>However, the problem here is that there are not enough hypnotherapists out there for those who need it. Sessions are often long and costly, something that will not change anytime soon.</p><p>We know that this is a problem that not a hammer nor a needle can solve, so we’re building a tool that can — introducing Lana.</p><h3>Using Hypnotherapy to Our Advantage</h3><h4>Lana</h4><p>This is where Lana, our team’s moonshot idea, comes in (<a href="https://www.linkedin.com/in/jasmine-wang-09460a173/">Jasmine Wang</a>, <a href="https://www.linkedin.com/in/max-holschneider-8715121b7/">Max Holschneider</a>, and <a href="https://www.linkedin.com/in/allen-wang1536/">myself</a>). It will consist of an <strong>EEG-based BCI</strong> to provide biofeedback on the hypnotic state, as well as an <strong>app companion</strong> that provides the audio stimuli. Lana will take advantage of the varying effects of hypnotherapy to <strong>provide personalized and effective treatment for each individual’s alcoholic addiction.</strong></p><p>There is no product out on the market that could even compare with our method; The only things that exist are prerecorded sessions that are meant to be one size fits all. These apps have very little scientific backing and even less efficacy rates. The point being, our app will be miles ahead of anything that is out there.</p><p>Now, how does this all work?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/983/0*tlXSClB8ERKWEBp6.jpg" /></figure><h4>Step 1: Getting Into a Trance</h4><p>Traditionally, hypnosis can be classified into two categories: Hypnotic Induction, in which a person is hypnotized by an external person, and Self-hypnosis, where a subject listens to an audio recording on a tape.</p><p>However, Lana will utilize machine learning content generation that can allow for fluid hypnosis sessions both not alone and not with a hypnotist, but somewhere in between. We will make use of personalized script and delivery in order to come up with the most effective means.</p><p>To get the user into a trance, Lana will give out traditional trance-inducing commands. It will utilize different combinations of words and delivery in order to get the user into a hypnotic state as quickly as possible (as determined by the biofeedback data given from the BCI).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*F6gaPKA43W4nU2nW.jpg" /></figure><h4>Step 2: Implementing Suggestions</h4><p>Now that the user is effectively under a hypnotic state, we can begin implementing the suggestions.</p><p><strong><em>Audio Script Generation</em></strong></p><p>To explain how we will audio content works and will improve, let us pretend we were making a horror movie script for <em>Friends</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BxCtoS1GxLZfK_98sXueBQ.png" /></figure><p>First, we will build a dataset of a large number of script lines based on which character was speaking, which we will then put through a GPT-3 text generation model.</p><p>This will then turn into something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8zJA_2Ta8KPq_UWl9q8Npw.png" /></figure><p><em>Voila</em>, our horror-themed Friends script!</p><p>We will do the same thing mentioned above, but just with calm, hypnotic suggestions. We will create a sizeable database of hypnotic suggestions categorized based on the stage (induction, suggestion, awakening) and train a script generation model to do the rest for us.</p><p>We will utilize GPT-3 to write out custom scripts based on the type of alcohol addiction people want to overcome. For example, let’s say I wanted to stop drinking whenever I’m out with friends, the text generation script could create a custom script for that.</p><p><strong><em>Script Audio Delivery</em></strong></p><p>Scriptwriting is only half the battle, however. Once we have the script, we need to figure out the best content delivery. Now, we could just use a simple text to speech, but doing so could mean that your subconscious would just reject the suggestion, as we saw with the <a href="https://centerhistorypsychology.wordpress.com/2017/02/23/the-psycho-phone/">psychophone</a>.</p><p>We will program the ML model with a series of different factors, including audio volume, tone inflection, speaking speed, and background sound effects. All of these different attributes will, in the right proportions, have a positive effect on the user’s experience.</p><p><strong><em>Improvement and Training</em></strong></p><p>We will use biofeedback (EEG-based BCI) as well as conscious feedback in order to determine the effectiveness of our script / delivery. Conscious feedback will include data such as the number of drinks a user consumed in a week, or the number of temptations a user felt throughout a day. Using a text-to-speech voice synthesizer that will relay our script with the aforementioned qualities mentioned above, we can test out different combinations of attributes to custom tailor and make our suggestion delivery methods as effective as possible for each user.</p><p>Should the script at any time cause the user to awaken out of their trance, Lana will play the personalized trance-inducing track to ease them back into a trance before continuing with the script.</p><h4>Step 3: Continuation of Sleep</h4><p>One worry of learning in your sleep is that it loses its effectiveness since you are concentrating on your brainwaves. However, once the script is over, the app will go quiet, and the <a href="https://ezinearticles.com/?How-To-Pull-Someone-Out-of-a-Hypnotic-Trance&amp;id=5685106">user will naturally continue to sleep as if never disturbed from his or her sleep.</a></p><p><em>Typically, it takes around 8–12 normal hypnotherapy sessions to see results, and it would be recommended for users to use Lana daily. Even after progress starts to occur, users should keep on using Lana in order to prevent relapses. If the user does not see results after 3 weeks of consistent use, they should contact a medical professional for more help.</em></p><h3>Current Limitations</h3><p>As mentioned earlier, this is all still a moonshot idea, so there are a few issues that must be addressed before this can be feasible</p><ol><li>GPT-3 does not understand how to include high-level context yet. Many hypnotic suggestions require specific contextual cues (affirmation or dissuasion, harsh or soft, vernacular or King’s English), and so this is a crucial part of making our script.</li></ol><ul><li>However, basic context functionality is already available, and <a href="https://analyticsindiamag.com/10-must-read-technical-papers-on-nlp-for-2020/">NLP is one of the fastest-growing fields of machine learning</a> with much research looking into building better context models. Thus, we believe that this can be accomplished within 3–5 years.</li></ul><p>2. The practice of network analysis is still in its beta phase, and thus the biofeedback may not be as accurate as would be hoped for</p><ul><li>However, more research is being done on <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6221897/">the practice of network analysis</a>, and basic procedures have already been tested, leading us to believe that it can be completed within the coming years.</li></ul><p>3. The necessary BCI’s are a bit clunky and not suitable for convenient portable use.</p><ul><li>However, like all technology, BCI’s are rapidly getting smaller and smaller as more research is being done on it. We believe that more powerful non-invasive EEG-based BCIs can eventually be packaged in the form of a headband (similar to that of the Muse headband) within 2–4 years.</li></ul><h3>Vision For The Future</h3><p>Clearly, there are still issues that must be addressed. Yet, if we can get this technology working, the potential could be limitless.</p><p>In this moonshot, we are only specifically thinking of curing alcohol addiction, but <em>why stop there</em>?</p><p>We could help solve so many problems, ranging from anxiety to phobias to eating disorders. In fact, why even stop at simply curing problems, why not help to create better habits, such as waking up earlier or becoming confident?</p><p>The possibilities are truly endless, and we envision that Lana could eventually be used to form / break any habits as desired.</p><p>Lana has the potential to be the next universal tool, and it can, and will, help to<strong> further society’s growth at a faster rate than ever seen before.</strong></p><p>For further information, check out our website: <a href="http://thelanaapp.com/">http://thelanaapp.com/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e35096bb645" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/treating-alcohol-addiction-with-personalized-hypnotherapy-e35096bb645">Treating Alcohol Addiction With Personalized Hypnotherapy</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Reinforcement Learning]]></title>
            <link>https://medium.com/swlh/reinforcement-learning-cb9de05fb60?source=rss-d789c5c45b7a------2</link>
            <guid isPermaLink="false">https://medium.com/p/cb9de05fb60</guid>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Allen Wang]]></dc:creator>
            <pubDate>Mon, 16 Nov 2020 06:50:58 GMT</pubDate>
            <atom:updated>2020-11-28T10:09:17.705Z</atom:updated>
            <content:encoded><![CDATA[<h3>Crash Course: Reinforcement Learning</h3><p><em>A short high-level introduction (without all the complicated math) to Reinforcement Learning</em></p><p>What is the process of training a dog to sit like? Well, your dog may initially be completely untrained and have no idea what to do.</p><p>You might tell him to sit, and the dog might start barking. You scold him and tell him and tell him to sit again, but this time he starts wagging his tail. Once again, you scold him. You continue to try and tell your dog to sit, and finally, on the 27th try, he sits! You give him a treat and a word of praise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/730/1*1hPgvx18d4ZU03__EWmmhA.jpeg" /></figure><p>As you keep up this cycle of scolding and praising, your dog eventually learns to sit when you tell him to. <em>Voila</em>! You have just demonstrated <strong>reinforcement learning</strong> with your dog!</p><h4>What exactly is Reinforcement Learning?</h4><p>Reinforcement learning is essentially where <strong><em>an agent is placed in an environment and is able to obtain rewards by performing certain actions</em></strong>. The agent’s only goal is to maximize the amount of reward it can get.</p><p>In the dog training example, your dog serves as the agent, and the rewards were your words of praise and treats.</p><p>Reinforcement learning works by using trial and error (it can be very tedious, as seen from the dog training) from its own actions and experiences.</p><h4>Reinforcement Learning vs Supervised/Unsupervised Learning</h4><p>Reinforcement learning is a subset of<em> machine learning</em>, as are supervised and unsupervised learning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aypfrNvKTlOi5q5gdfUz5Q.jpeg" /></figure><p>The main difference between reinforcement learning and supervised learning is <em>sequential decision making</em>. While in supervised learning, the actions the agent makes does not affect the future, in reinforcement learning, every single input depends on the previous action the agent made.</p><p>The main difference between reinforcement learning and unsupervised learning is their<em> goals</em>. In unsupervised learning, the main goal is to find structure within a given dataset, whereas in reinforcement learning, the goal is to find a course of action that would maximize the total reward for the agent.</p><p>All 3 of these fall into the massive umbrella of machine learning in which agents learn from data.</p><h4>Markov Decision Process</h4><p>The Markov Decision Process is <strong><em>the mathematical framework that describes the environment of a reinforcement learning model.</em></strong> In reinforcement learning models, all future states depend only on the present state, which means that they are a Markov Process. Reinforcement learning is a technique that attempts to learn an MDP and find the optimal policy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*dY_Fo9_OPnv1HrhK.jpg" /></figure><h4>Common Terminology:</h4><ul><li><strong>State</strong> is essentially what the agent observes in its environment at a certain moment</li><li><strong>Actions</strong> are the possible moves that the agents can perform in the environment</li><li>The<strong> Reward</strong> is what the agent receives if it achieves a desirable result</li><li><strong>Discount</strong> is an optional factor that determines the importance of future rewards relative to now; It can range from 0 to 1.</li><li>The <strong>Value</strong> of a state is the expected long-term return (may include discount for the state)</li><li><strong>Policy</strong> is the strategy that the agent employs to determine the next action. The optimal policy is the one that maximizes the amount of reward expected to receive.</li></ul><h4>Maze Game Example</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/628/1*A8XR3OQuvliiz7AmyAKDbg.png" /></figure><p>Let’s say the agent was the robot and the maze was the environment. The state would be the position of the robot at any point in time. It would utilize a policy to determine which path to take. The actions would be going left, right, up, or down. We could award the robot +1 point for hitting an empty square, -1 point for hitting a wall, and+100 points for reaching the exit.</p><p>At first, the robot starts off with no experience at all, so it has completely random movements. However, as it starts to learn the values of each state, it begins to become smarter and smarter, finally completing the maze.</p><h4>Exploitation vs Exploration</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/628/1*ieuobciOCs3O5nqopXHzNw.png" /></figure><p>Let’s revisit the maze example. Let’s pretend that the robot initially chooses to go to the right, thus assigning a value of 1 to the white space to the right of it. Then it goes on and ends the episode. When the robot starts again, it now knows that the state of the space to the right has a value of 1, while the state of the space on top has a value of 0. Thus, <em>it will always go right no matter what!</em></p><p>This brings us to the choice of <strong>exploitation vs exploration.</strong> The robot never actually had a chance to explore all the other options; It simply chose the state that had the highest value. This is known as the <strong>Greedy Policy</strong>, <em>where the agent always picks the highest value.</em></p><p>One option here is to “exploit” the agent’s previous knowledge, <em>in which it always picks the state that will give it the highest reward</em>. The other option would be to <em>“explore” the other states to see if they would potentially give a higher reward.</em></p><p>Depending on the situation, both have their advantages. If you needed to minimize the amount of loss, exploitation would work well for you to use previous experiences of what states received rewards. If you simply wanted to find the best possible method, without simulations that did not have any restrictions, exploration would be best.</p><p>Normally, a combination of both exploitation and exploration is used depending on the problem that needed to be solved (this can be changed as the project progresses).</p><h4>Episodic vs Continous</h4><p>A reinforcement learning model can be either episodic or continuous.</p><p>Episodic simply means that there is a <strong>“terminal” condition for the game</strong>, whether that be winning or losing. An example of a program that would require episodic reinforcement learning would be the game pong. In pong, the simulation resets every time the agent wins or loses the game (first to 11 points).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/460/0*Y-zKkyAyZAb7ovTa" /></figure><p>Continuous means that there is <strong>no end condition for the game</strong>, and that the model will just keep on running until stopped. For instance, a reinforcement learning model applied to the stock market would keep on going until it is manually terminated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yv2zBpU1uZCaIn_I.jpg" /></figure><h4>Monte Carlo vs Temporal Difference Learning Methods</h4><p>In the maze example, the reinforcement learning model could’ve used either the monte carlo or temporal difference method.</p><p>The Monte Carlo method waits until the end of the episode, then <strong>checks the cumulative reward that it has received</strong>. It then calculates and updates the the expected award for each state at the end.</p><p>The Temporal Difference method <strong>updates the value of each state after each time step instead of at the very end.</strong> Thus, it continually receives feedback from rewards and updates its guesses of each value of the state.</p><h3>Different Approaches to Reinforcement Learning</h3><h4>Value Based</h4><p>In a value-based reinforcement learning, we want to find the maximum value function.</p><p>The <strong>value function is a function that tells us the amount of reward that an agent can expect to get in the future at each state</strong> (That is what we were using for the maze example). With this learning, the agent will always pick the state that has the highest reward.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/748/0*iFMPYHbV_06-qr4u.png" /></figure><p>In this example, the agent will start at -7, then continue on to -6, -5, and so forth until it reaches the goal, as those states have the largest values. Once you find the optimal value function, the optimal policy is able to found from it.</p><h4>Policy Based</h4><p>In a policy based reinforcement learning, the agent is essentially told where to go by the policy function. The policy function is a function that tells how the agent will make a decision.</p><p><strong>Policy functions usually start off random and with a value function that corresponds to it.</strong> It then finds a new value function and improves its policy. It keeps going until it finds the optimal policy and value function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/748/0*6Gy6sruPlqeHyyHb.png" /></figure><p>As shown, the policy function essentially tells the agent the best direction to go.</p><h4>Wrap-up</h4><p>So that’s it! You just completed a high-level overview of the components of Reinforcement Learning. Just to recap:</p><ul><li>RL is essentially where an agent is placed in an environment and is able to obtain rewards by performing certain actions.</li><li>RL is different from supervised/unsupervised learning</li><li>RL attempts to find the most optimal policy of the Marko Decision Process</li><li>Exploitation picks the most optimal choice from is known, while exploration picks a random choice that may not be considered optimal at the moment</li><li>Episodic means that the RL model has a designated win/loss, while continuous means that the RL model will keep running until manually stopped</li><li>Monte Carlo method receives reward and updates value function at end of episode, while TD makes guesses to improve the value function at the end of each time step</li><li>Value Based finds optimal value function then derives optimal policy function from it, while policy based continuously updates its policy function to find the most optimal policy and value function</li></ul><p>Hope you learned something from this article!</p><p>Feel free to reach out to me on I<a href="https://www.instagram.com/allenn.wangg/">nstagram</a>, <a href="https://www.linkedin.com/in/allen-wang1536/">LinkedIn</a>, or email allenwang1536@gmail.com!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cb9de05fb60" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/reinforcement-learning-cb9de05fb60">Reinforcement Learning</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Power of Habit]]></title>
            <link>https://allenwang1536.medium.com/the-power-of-habit-96f8bbefea89?source=rss-d789c5c45b7a------2</link>
            <guid isPermaLink="false">https://medium.com/p/96f8bbefea89</guid>
            <category><![CDATA[habits]]></category>
            <category><![CDATA[self-awareness]]></category>
            <category><![CDATA[change-your-life]]></category>
            <category><![CDATA[power-of-habit]]></category>
            <category><![CDATA[self-improvement]]></category>
            <dc:creator><![CDATA[Allen Wang]]></dc:creator>
            <pubDate>Mon, 19 Oct 2020 05:54:37 GMT</pubDate>
            <atom:updated>2020-10-19T05:54:37.927Z</atom:updated>
            <content:encoded><![CDATA[<p>Don’t lie. We’ve all been here. “Yeah bro, I’ll finish it later don’t worry about it.” The next thing you know, it’s 11:37pm and you’re rushing to meet the task deadline. Why is that we do this? Is there a way to change this terrible habit of ours?</p><p><strong>Our life is driven by our habits.</strong> Everything we do, from the smallest of the small to the biggest of the big is, in large part, controlled by our unconscious habits. The crazy thing is that most of the time we don’t even realize it! Most have never even considered altering or transforming their habits for the better. Luckily, this article will go over exactly how habits work and how we can use that understanding to our advantage.</p><p>I’ll be covering one of the main points of the book, <em>The Power of Habit, </em>by Charles Duhigg.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/324/1*sevPG7MyVeBrM78xSxQuJA.jpeg" /></figure><h3>Why do we even have habits in the first place?</h3><p>Our brains are super lazy. They are constantly looking for ways to find the path of least resistance, to find the most efficient method to accomplish something. That’s why we have habits, which help <em>lighten the strain on our brain</em> by helping us to carry out our numerous daily tasks that we don’t even think about anymore.</p><p>There are <em>hundreds</em> of behavioral patterns that are carried out by us every day. Some are simple: brushing our teeth, washing our face, tying our shoes. Others can be more complex: getting dressed, making lunch, writing an article.</p><p>Some are so complicated, that it’s incredible that we can do them without thinking! Let’s say you’re backing your car out of the driveway in the morning. When you first started to learn this, it took a great amount of concentration in order to be successful: You had to open the garage, unlock the car door, adjust the seat, insert the key in the ignition, turn it clockwise, move the mirrors, check the obstacles, put your foot on the brake, move the gearshift into reverse, remove your foot from the brake, mentally estimate the distance from the garage to the street while you simultaneously keep watch for oncoming traffic, calculate distances of your car to the trash can, hedges, bumper, and all the while telling your mom to <em>please </em>calm down and assure her that you’re not going to crash.</p><p>Whew, that was <strong>a lot</strong>.</p><p>However, when you back your car out of the driveway now, you hardly have to think about any of these things anymore. This is all because of <strong>the habits that our brain has formed.</strong></p><p>As we got more and more used to this morning process, our basal ganglia (a part in our brain) began to take over, identifying the habit that we’ve gotten so used to and executing the pattern for us. Once this occurs, we’re free to think about other urgent thoughts, such as how we’re going to pass the math test we didn’t study for last night (oops).</p><h3><strong>The Habit Loop</strong></h3><p>Habits, if you break them down, are a pretty simple process. It involves a three-step process that consists of a <strong>cue, a routine, and a reward.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1022/1*H2ZuQw6F-CjwStwFHfe6Fg.png" /></figure><h4>Cue, Routine, Reward</h4><p>To begin with, <strong>a cue is a trigger that causes our brains to go into an automatic mode and call upon an action</strong>. For instance, if a rat hears a meow, the cue, it automatically knows to activate its “flight” habit. If a dog sees a bird, the cue, it automatically activates its “chase” habit. Whatever it is, all habits start off with a trigger cue that begins it.</p><p>After the cue is the routine. <strong>The routine is what action our brain tells us to do once the cue is triggered</strong>. For instance, this may be the whole process of backing a car out of the driveway after getting into the car. Or it could be your whole morning routine after your alarm sounds.</p><p>Finally, the habit loop ends with a reward. A reward is returned to the brain after every habit is carried out, and this is <strong>what the brain uses to determine if the habit is worth saving down or not</strong>. For instance, if you decide to exercise one day, then later reward yourself with a quick snack, your brain might decide that this habit (exercising) is worth storing in order so that you get the award (the snack) again next time.</p><h4>Effect of Habit Loop</h4><p>As you begin learning and storing these habits, your brain decides to <em>stop the decision-making process for these actions</em> which <em>allows you to divert your focus to another task</em>. Though this is beneficial for conserving energy and effort, it also makes it <strong>much harder for us to change our habits.</strong></p><p>Unless you intentionally fight<em> </em>against a habit and find new routines, a habit pattern <em>will be carried out almost automatically</em> every time you encounter a cue. This explains why it is so hard to create a good exercise habit, for instance, or to stay on a good diet.</p><p>Once we begin to develop the habit of sitting down on the couch after work rather than going out for a light jog, those patterns remain inside our heads, and it can be difficult to try and fix them.</p><p>However, using the same principles you just learned about the habit loop, we can learn to overpower these behaviors and create new positive habits!</p><h3>The Craving Brain</h3><p>Before we start on changing our habits, we need to focus on one more crucial aspect of the habit loop that wasn’t mentioned before. Cravings! As habits begin to build and get stronger and stronger within your mind, you begin to start to <strong>crave and anticipate the rewards before you receive them.</strong></p><h4>How It Works</h4><p>When you first start to build a habit, you go through the normal cue routine reward cycle. As this cycle gradually builds up, however, you start to <em>anticipate</em> the reward more and more, to the point where your brain starts associating the cue <em>with</em> the reward without actually receiving the reward.</p><p>Researchers have discovered that once this habit and craving are firmly ingrained in one’s brain, it can be hard to change. The anticipation may become so overwhelming that it becomes all a person can think about, which leads to addiction of all sorts (both good and bad).</p><blockquote>This explains why these habits are so powerful:<strong> They create neurological cravings.</strong></blockquote><h4>Cinnabon Kiosk Placement</h4><p>One fascinating way that this is employed in real life is with the placement of Cinnabon stores<em>.</em> Normally, food vendors try to position their kiosks in or near the food court. However, Cinnabon stores do the exact opposite, and try to put their stores away from all the other food stalls. This way, the smells of the other stores are not distracting from theirs.</p><p>Why do they do this? <strong>To create a craving.</strong></p><p>Let’s say a consumer is walking down a hallway, maybe just trying to pick out a present for his mom. As he is walking, the scent of the cinnamon rolls wafts down his hallway and starts to subconsciously cause his mind to crave a roll. His mind begins to crave the reward of a roll (the sugar high) without him realizing it. By the time he finally rounds the corner and sees the Cinnabon store (cue), the craving is roaring in his mind, and he automatically goes to pull out his wallet to buy one. All this happened because of his habit loop.</p><p><strong>Cravings are what drives the habit loop.</strong> The basic habit loop is employed when a person is just starting to learn and memorize the habit. The habit only truly emerges once a person begins craving the reward once they see the cue. <em>Once that craving exists, the person will react and carry out their habit loop automatically.</em></p><h3>The Golden Rule of Habit Change</h3><p>Great, you might say. Now that I know all about habit loops and cravings, how do I change my habits for the better? It’s a pretty simple process. To change an old habit, you have to <em>first address the old craving</em>. You can do so by <strong>creating a new habit loop with the same cues and rewards as before, but with a different routine</strong>. Researchers have discovered that habits are malleable when employing this technique.</p><h4>Mandy’s Nail-Biting Problem</h4><p>One example of this principle in play was with a graduate student named Mandy. She had long struggled with nail-biting and had gone to therapy to try and fix this problem. After talking about it, she had discovered that the reward from her nail-biting habit was a physical simulation that she had come to crave.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/810/1*Oq102s63qGCDNT0n_2IqvQ.png" /><figcaption>Mandy’s old habit loop that caused her to bit her nails any time she received a cue in her fingertips.</figcaption></figure><p>Her therapist then told her that whenever she felt tension in her fingertips (cue), she should look for a quick physical stimulation, such as rapping her knuckles on the table or rubbing her fingers on her arm. Consequently, her nail-biting habit was gone after a month.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/742/1*5AS-Syp0u-__tRaXGqBevw.png" /><figcaption>Mandy’s new habit loop that replaced the old nail-biting routine with a new better routine.</figcaption></figure><p>This was a perfect example of the golden rule of habit change in place. In both habit loops, Mandy kept the same cue (tension in fingertips) and reward (physical stimulation). <em>Only the routine changed</em><strong>,</strong> from nail-biting to another form of quick physical stimulation. Because of this, Mandy was able to<strong> </strong>replace her old negative habit loop with a better one.</p><h4>The Habit Loop</h4><p>As you can see, the process itself is pretty simple if you want to change a habit. <strong>You discover what reward you are craving from the habit, and replace the old routine with a new and better one that provides the same reward.</strong></p><p>Of course, you will need to genuinely believe change is possible, and be intentional about working on overcoming your bad habit. But now knowing about how a habit loop works should allow you to become more aware of your own choices.</p><p><strong>The Golden Rule of Habit Change:</strong> The simple rule that will allow you to form positive habits whenever you need them. What habits will <em>you</em> change?</p><h4>Recap:</h4><ul><li>Habits reduce the strain on our brain</li><li>The habit loop is driven by cue, routine, reward loop</li><li>Cravings drive the habit loop</li><li>We can replace a bad habit with the Golden Rule of Habit Change, in which we form a new habit loop with the same cues and rewards as before, but with a different routine</li></ul><p>Feel free to reach out to me on I<a href="https://www.instagram.com/allenn.wangg/">nstagram</a>, <a href="https://www.linkedin.com/in/allen-wang1536/">LinkedIn</a>, or email allenwang1536@gmail.com!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=96f8bbefea89" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[An Incoming Genetic Revolution]]></title>
            <link>https://allenwang1536.medium.com/an-incoming-genetic-revolution-573d60ceddb2?source=rss-d789c5c45b7a------2</link>
            <guid isPermaLink="false">https://medium.com/p/573d60ceddb2</guid>
            <category><![CDATA[gene-editing]]></category>
            <category><![CDATA[crispr]]></category>
            <category><![CDATA[prime-editing]]></category>
            <dc:creator><![CDATA[Allen Wang]]></dc:creator>
            <pubDate>Sun, 24 May 2020 17:05:47 GMT</pubDate>
            <atom:updated>2020-11-29T01:21:53.668Z</atom:updated>
            <content:encoded><![CDATA[<h4>Gene therapy is ascending to new heights with the development of prime editing</h4><p>We’re all guilty of having envied the characteristics of our friends. Whether it be their eye color, height, or muscular build, we can all agree there are features of our friends that we wished we possessed. Alas, we’ve been raised into a world where we’re born with traits that are unalterable, determined by our complex and rigid genetics. But wait.</p><p><em>What if there was a way we could alter our genes, to become the perfect version of ourselves that we’ve long wished for?</em></p><p>That’s where gene editing comes in. Though we may still be a ways away, fully genetically engineering humans is now closer than ever with the arrival of <strong>prime editing</strong>, a new way of genetic modification.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*3COhMAVgS7Q3Vq979iM3pQ.jpeg" /></figure><h4>A <em>new</em> way? What did we have before?</h4><p>Before prime editing, we had a system known as CRISPR, short for clustered regularly interspaced short palindromic repeats. In order to understand prime editing, we must first take a look at the CRISPR system.</p><p>This system was adapted from bacterial immune systems defending against invading viruses. When new viruses enter a bacteria cell, the CRISPR system creates a class 1 Cas protein that breaks apart the viral DNA, and stores a piece of the DNA segment into the CRISPR arrays. This essentially allows the immune system to “<em>memorize</em>” this type of virus DNA.</p><p>In the event that viruses assault once more, a Cas enzyme complex searches for the virus’s DNA match in the CRISPR array. From this located DNA, an RNA section, known as CRISPR RNA (crRNA), is produced and used by the complex to kill the viral DNA.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*Dh9a1g6T4P8trIcBworrig.jpeg" /><figcaption>CRISPR Locus example. The spacer DNAs are the viral DNAs the bacteria have encountered, with the repeats separating the different viral DNAs. The CAS genes are what produces the enzymes, the Cas enzyme complex and the class 1 Cas protein. The leader is simply a segment of DNA that holds the two parts together.</figcaption></figure><h4>It seems to me that you’re talking an awful lot about immune systems… where does the gene-editing stuff come in?</h4><p>Scientists observed this phenomenon occurring in bacteria, and some attempted to adapt and utilize this system to our advantage. The most famous adaptation was CRISPR-Cas9.</p><p>This system contains only one Cas9 protein (unlike the bacterial system which used two different ones), as well as something known as guide RNA (gRNA). The gRNA has a predesigned sequence that matches a section of the DNA segment, which helps the Cas9 protein locate the targeted part of DNA. After the Cas9 protein locates the DNA segment, the Cas9 protein nuclease<strong> </strong>is used to cut the DNA into two parts. This ensures that the genome is cut at the correct location.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BtXMhHIUd7s9IFKhiyMgmw.png" /><figcaption>The Cas9 protein is the white structure and the gRNA is the gray strand with blue dots. The red strand is the targeted DNA segment. The two edges of the Cas9 protein nuclease on the DNA segment is used to separate the DNA into two pieces.</figcaption></figure><p>Now, since the Cas9 protein complex has cut and separated the DNA, scientists are able to put their desired DNA addition between the two separated pieces of DNA. This effectively modifies the genetic code, and can be used to treat a variety of genetic problems, such as cystic fibrosis or hemophilia.</p><h4>Wow! This CRISPR-Cas9 system sounds great! Why do we need a new system?</h4><p>Yes, the CRISPR-Cas9 tool does sound like the ultimate gene-editing tool, at least in theory. However, as scientists have discovered, the CRISPR-Cas9 tool is <strong>prone to many errors and unintended effects</strong>. Because it cuts through both strands of DNA, the repair process of the DNA segment <strong>can introduce dangerous mutations</strong>, as the DNA repair systems are likely to insert random insertions or deletions when repairing the genome.</p><h4>How is prime editing able to fix this?</h4><p>In order to understand prime editing, you need to understand where the idea came from. Prime editing originated from the idea of <strong>CRISPR base editors</strong>. In the base editor system, the Cas-9 protein <strong>doesn’t cut through the DNA strands, which reduces the occurrences of unwanted mutations</strong>. It still uses the original CRISPR targeting system, but<strong> converts only a single nucleotide into another, instead of a whole strand of DNA</strong>. Though safer than the traditional CRISPR-Cas9 cut and paste system, it has heavy limitations on the genetic diseases it can treat. Thus, scientists raced to find a better alternative.</p><p>That’s where prime editing comes in. Prime editing ensures not only safety but also offers treatment to a wide range of genetic diseases.</p><p>Prime editing differs from the traditional CRISPR system by using an altered version of both the Cas-9 protein and gRNA. This method uses a Cas-9 nickase fused with a reverse transcriptase, known as the prime editor. The new guide, known as pegRNA, now contains an additional RNA template as well as a primer binding sequence (PBS). The RNA template is used by the reverse transcriptase to create the desired DNA segment, and the PBS is used to bind the pegRNA onto the DNA segment after it has been snipped (not to be confused with the gRNA target sequence).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e0Rpeieh_ccj45-m_k86-w.png" /><figcaption>*Image by Synthego</figcaption></figure><h4>How does this new prime editing system work?</h4><p>Similarly to before, the PE:pegRNA complex first locates the target DNA segment. However, the Cas9 nickase nicks only one of the strands loose, creating a flap of DNA that the PBS can bind to and hold in place. This allows the reverse transcriptase to attach to the flap and create a new genetic sequence by reverse transcribing the edited RNA template, which replaces the original flap on the DNA strand. The prime editor now removes the flap’s corresponding segment that is on the opposite strand of the original DNA, which is still unedited at this time. This triggers the repair system to fix the damage by using the newly placed, synthesized gene as a template. Thus, both strands of DNA are effectively edited at the end of this process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DkZTzvdHiztsrqA-jmJgDQ.png" /><figcaption>The blue section is the PBS that binds and holds the reverse transcriptase to the DNA flap. The yellow shape is the reverse transcriptase that reads the desired RNA segment(red) and creates the new, synthesized DNA (purple). The new DNA (purple) eventually replaces the original(green flap) in filling the open section of the original DNA segment. The unedited strand is then removed, which causes it to be repaired by using the new DNA segment as a template. *Image by Synthego</figcaption></figure><h4>Why is this method better than before?</h4><p>As previously mentioned, prime editing works by nicking one strand of DNA while the original CRISPR system cuts through both strands. Cutting only one strand prevents the DNA from automatically activating its repair system, which <strong>reduces the chance of genetic mutation</strong>s.</p><p>Additionally, the prime editing (PE) system requires three pairing events to operate, as opposed to the single pairing system used in CRISPR-Cas9. This allows for <strong>increased accuracy and reliability when replacing the gene sequence</strong>. In fact, prime editing’s reduced off-target effects compared to the traditional CRISPR-Cas9 system <strong>allows it to make single-nucleotide edits in DNA</strong> <strong>that even base editors are unable to</strong>.</p><p>Besides being more accurate and less prone to errors, the PE system has much more versatile applications. Its edits now include <strong>12 possible combinations of single base replacements</strong> against the <strong>4 that base editing is capable of</strong>. This new system can now swap DNA letters into any other, as well as delete or insert letters in a genome, doing so with minimal damage to the DNA strands.</p><p>Another benefit of this new editing tool is <strong>its ability to operate in cells that can’t divide</strong>, something that the traditional CRISPR system has been unable to do. This allows PE to correct genetic mutations caused in cells that don’t divide, such as those in the nervous system.</p><h4>Sooo…what’s the impact of all this?</h4><p>Prime editing provides us with a tremendous advantage that base editing and CRISPR cannot. With the PE system’s more precise replacement technique and smaller error rate, scientists are able to <strong>better control their DNA replacement segments than the original CRISPR-Cas9</strong>. The additional versatility also allows the prime editing system to <strong>target more types of genetic diseases</strong> that both CRISPR and base editing are unable to fix successfully.</p><p>All this has led scientists to believe that prime printing can theoretically <strong>correct at least 89% of all human genetic diseases</strong><em>. </em>If properly developed, the PE system could eventually cut and replace the cancer markers in DNA, and become the long-awaited cure for cancer.</p><p>Moreover, prime editing will be able to do all these new treatments at a much lower cost. For instance, cancer treatments that often drain a patient’s financial state could later be treated using a (relatively) cheap treatment of prime editing. PE could also treat long-term disabilities such as those with Alzheimer’s or special needs, thus reducing and maybe even removing the costs needed to take care of them altogether. This new editing technique could lead to an <strong>enormous decrease in healthcare costs</strong>, effectively redefining how the healthcare system works.</p><h4>Wow! We’re basically invincible now!</h4><p>Not quite. Though it has lots of potential, prime editing is still in its early stages, with much more testing needed to validate usage. So far, prime editing has only been tested in a small number of genetic diseases, while CRISPR has been proven through years of clinical research. In addition, prime editing has thus far been unable to edit larger sections of genetic material, which are needed for conditions such as hereditary heart disease, so the CRISPR system is still necessary for those types of diseases.</p><p>However, in due time and with lots more tests, prime editing may very well become the future of all genetic disease treatment plans. With the ability to accurately modify genetic structure with minimal risks, the technique has the potential <strong>to provide limitless treatments to genetic diseases</strong>, and eventually even address therapeutic treatments for medical conditions. Providing all this <strong>at a cheaper cost</strong>, the method of medical care will have to be re-evaluated in the coming years. As such, prime printing may well become <strong><em>the start of a genetic revolution.</em></strong></p><p>Feel free to reach out to me on I<a href="https://www.instagram.com/allenn.wangg/">nstagram</a>, <a href="https://www.linkedin.com/in/allen-wang1536/">LinkedIn</a>, or email allenwang1536@gmail.com!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=573d60ceddb2" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>