Deep Reinforcement Learning Nanodegree Program: What You’ll Learn
Enrollment is now open for our Deep Reinforcement Learning Nanodegree program!
Deep Reinforcement Learning is the hottest research field in artificial intelligence, and the closest we’ve yet come to developing AI that can learn and develop like a human does! While there have been many advances over the last couple of years, the field debuted on the global stage when DeepMind’s AlphaGo defeated the world champion Go player Lee Sedol. More recently, researchers trained a team of AI agents to defeat human teams at Dota 2, one of the most complex esports games in the world.
In our new Deep Reinforcement Learning Nanodegree program, you’ll harness the most cutting-edge AI techniques to train your own intelligent agents!
With deep reinforcement learning, your agents will learn for themselves how to perform complex tasks through trial-and-error, and by interacting with their environments. These techniques have a number of real-world applications, including robotics, machine learning, personalized education, healthcare, finance, advertising, and conversational bots (or chatbots).
The possibilities of this world-changing technology are truly endless, and if you want to be at the forefront of this transformative field, then the Deep Reinforcement Learning Nanodegree program is for you!
Hands-on Learning
We collaborated with Unity and the NVIDIA Deep Learning Institute to build a world-class program in which you’ll experience a balance of theory and practical application, and which enables you to explore compelling challenges in fields ranging from gaming to finance to robotics.
Over the course of the program, you’ll implement several deep reinforcement learning algorithms using a combination of Python and PyTorch, to build projects that will serve as GitHub portfolio pieces to showcase your mastery of this advanced field.
All of the projects will use the rich environments from the Unity ML-Agents toolkit, and you’ll also solve many classic benchmarks using OpenAI Gym. Using the simulations from Unity and OpenAI, you’ll write your own code to train a spaceship to land and teach a robot to walk, among other complex tasks!
We’re also excited to release the official GitHub repository of the Nanodegree program, which contains a comprehensive collection of implementations of many of the algorithms that we will cover.
What You’ll Learn
This program is a four-month, hands-on introduction to this transformational technology, designed for those who have at least an intermediate Python background and some beginner experience with neural networks.
The curriculum is comprised of four sections, and features three projects.
1. Introduction to Deep Reinforcement Learning
We’ll start off with a simple introduction to reinforcement learning. You’ll learn how to use the Markov Decision Process framework to define real-world problems so that they can be solved with reinforcement learning.
Then, you’ll implement classical methods such as SARSA and Q-learning to solve several classic reinforcement learning benchmarks. You’ll explore how to use techniques such as tile coding and coarse coding to increase the complexity of problems that can be solved with traditional algorithms.
2. Value-Based Methods
In the second section, you’ll learn how to leverage neural networks when solving complex problems using the Deep Q-Networks (DQN) algorithm. You will also learn about modifications such as double Q-learning, prioritized experience replay, and dueling networks. Then, you’ll use what you’ve learned to create an artificially intelligent game-playing agent that can navigate a spaceship!
You’ll learn from experts at the NVIDIA Deep Learning Institute how to apply your new skills to robotics applications. Using a robotics simulator, you will train a rover to navigate an environment without running into walls.
Project 1: Navigation
In the first project, you’ll leverage neural networks to train an agent to navigate a virtual world and collect as many yellow bananas as possible while avoiding blue bananas.
3. Policy-Based Methods
In the third section, you’ll learn about policy-based and actor-critic methods such as Proximal Policy Optimization (PPO), Advantage Actor-Critic (A2C), and Deep Deterministic Policy Gradients (DDPG). You’ll also learn about optimization techniques such as evolution strategies and hill climbing.
Next, you’ll learn from experts from the NVIDIA Deep Learning Institute how to apply your new skills to the financial services industry for optimal execution of portfolio transactions.
Project 2: Continuous Control
In the second project, you’ll write an algorithm to train a robotic arm to reach moving target positions.
4. Multi-Agent Reinforcement Learning
In March of 2016, DeepMind’s AlphaGo stunned the world by defeating the world champion Go player Lee Sedol. As arguably the world’s most complex game, Go had long been a target for AI engineers, making this a history-making victory. In this section, you’ll learn about the skills that made this victory possible, and master techniques that can defeat AlphaGo!
You’ll also explore frameworks and techniques that can be used to train multiple, interacting agents, through a research area known as multi-agent reinforcement learning. Most of reinforcement learning is concerned with a single agent that must demonstrate proficiency at a task. In that scenario, there are no other agents. However, if we’d like our agents to become truly intelligent, they must be able to communicate with — and learn from — other agents. Multi-agent reinforcement learning has many real-world applications, ranging from self-driving cars to warehouse management.
Project 3: Collaboration and Competition
In the final project of the Nanodegree program, you’ll design your own algorithm to train a pair of agents to play tennis.
The Support You’ll Receive
Udacity offers you a wide array of support options to ensure you proceed through the program successfully. You’ll be able to connect with your fellow students by joining an online community where you can engage with our Community Manager, your fellow students, and even your instructors. You’ll also get detailed feedback on your project submission by one of our project reviewers.
To add an additional layer of dynamic support as you advance through the program, we’ve also assembled an outstanding roster of talent to be our Experts-in-Residence, including:
- Arthur Juliani, Deep Learning Researcher at Unity
- Avilay Parekh, Principal Machine Learning Engineer at Unity
- Melody Guan, Machine Learning Ph.D. at Stanford University
- Peter Welinder, Research Scientist at OpenAI
- Vincent Gao, Software Engineer (Machine Learning) at Unity
As a student in this program, you’ll have the opportunity to connect directly with these experts during weekly office hours sessions.
How to Enroll
We are now accepting new students to the Deep Reinforcement Learning Nanodegree program.
The program is comprised of a single four-month term. The tuition for the term is $999, paid prior to commencing your studies.
To learn more, you can also explore a Free Preview of this program. You’ll learn the basics of reinforcement learning, and train your own artificially intelligent agent with our in-classroom programming environment!
Deep Reinforcement Learning is one of the hottest research fields on the planet, and if your goal is to master this cutting-edge technology, then come learn Deep Reinforcement Learning and start building your skills today!
