Solving the Reinforcement Learning (RL) Frozen Lake Environment

Photo by Jakob Rosen on Unsplash

In this previous article I showed you how to setup the Frozen Lake RL environment:

Setting up the Frozen Lake Environment for Reinforcement Learning (RL)

I also discussed a number of ways to customize this environment, including the size of the grid world and controlling the stochasticity of the environment. In this article you will learn how to solve this environment using tabular Q-learning. See the training code below.

Training Code:

import gym
import numpy as np

# Create the Frozen Lake environment
env = gym.make('FrozenLake-v1', map_name="4x4", is_slippery=False)

# Initialize the Q-table with zeros
q_table = np.zeros((env.observation_space.n, env.action_space.n))

# Set the learning rate and discount factor
alpha = 0.1
gamma = 0.99

# Set the number of episodes to run
num_episodes = 100_000

# Store rewards for all episodes
total_rewards = np.zeros((num_episodes, 1))

epsilon = 1.0

decay_time = num_episodes // 10

# Run Q-learning
for i in range(num_episodes):
    if i!=0 and i % decay_time == 0:
        epsilon -= 0.1
        print(f"i = {i}", f"epsilon = {epsilon}")…