Feature thumbnail of the post containing, dog, cat, bone and robot icon and game screenshot
Feature thumbnail of the post containing, dog, cat, bone and robot icon and game screenshot

In my last post, I went through how we can create a custom maze adventure like environment using TF-Agents and how we can train, evaluate and visualize the performance of a DQN agent. In this post, I have added few more actions that our agent can perform and we will see how we can include that in the existing environment. …


Actions taken by agent/dog during 1 episode of gameplay and corresponding rewards and penalties received on each action step.
Actions taken by agent/dog during 1 episode of gameplay and corresponding rewards and penalties received on each action step.
Actions taken by agent/dog during 1 episode of gameplay and corresponding rewards and penalties received on each action step.

Deep Reinforcement Learning is a type of Reinforcement Learning algorithm which uses Deep Neural Networks that helps agents in making decisions. TF-Agents is a framework and a part of TensorFlow ecosystem that allows us to create such Deep Reinforcement Learning algorithms easily. In this blog I will explore basic of Deep Reinforcement Learning using TensorFlow Agents or TF-Agents in short by creating a simple custom game to illustrate the concepts and steps that goes in creating it using the framework.

Overview of the game

Reinforcement Learning is used in scenario where the system is required to make decisions based on the input they receive. To better understand this concept, let’s take an example of a custom game that we will create and let’s call it ‘Dog in a park’. The game is very simple, we have a dog who is the main player of this game and his name is Kiko. There are few challenges ahead for Kiko. He is in a section of a park where he isn’t supposed to be. But our Kiko is a very greedy dog, and he got there by following the smell of the bones lying on the ground. Every second he spend there is very risky and he needs to get out of that section as soon as possible. To make matter worst, there are already some robots present in that section to secure the area and if they see him, he is definitely in trouble. Now, the challenge for Kiko is, he needs to eat as much bones as he could find and as quickly as possible before reaching the exit and that too without being noticed by any of the robots. …


Image for post
Image for post
Fig 1: Image extracted from the original paper: https://arxiv.org/abs/1902.04502

Fast Segmentation Convolutional Neural Network (Fast-SCNN) is an above real-time semantic segmentation model on high resolution image data suited to efficient computation on embedded devices with low memory. The authors of the original paper are: Rudra PK Poudel, Stephan Liwicki and Roberto Cipolla. The code used in this article is not the official implementation from the authors but an attempt by me to re-construct the model as described on the paper.

Since the rise of autonomous vehicles, it is highly desirable that there exist a model that can process input in real-time. There already exist some state-of-the-art offline semantic segmentation models, but these models are large in size and memory requirement and requires expensive computation, Fast-SCNN can provide solution to all these problems. …


Image for post
Image for post

Recently we initiated Artificial Intelligence for Development or AID in short, to empower AI movement in Nepal. The idea is to develop AI talent in Nepal and connect them with domain experts from diverse fields such as health, education, agriculture, finance, transportation etc and create an environment to develop AI solutions to existing community problems in such fields.

This approach will develop AI ecosystem in Nepal and will address following issues:

  1. It will allow new AI learners to learn from practical problems in local community.
  2. AI developers will exercise their skills to develop solutions that are directly beneficial to the…


Couple of months ago, we, Developer Sessions team started a project for the blind community of Nepal. The project is Cash (Monetary Notes) recognition for visually impaired.

Here is Intel Software Blog article written by me about this project

In Nepal, the monetary notes (Cash) are not accessible for visually impaired individuals, there are no special markings in these notes to let them know what they are carrying. Some, with years of experience finally learned to recognize them while some still have to seek help of others to know the value of the note they are carrying.

We tried to solve this problem by creating a smartphone app. As most of the blind individuals uses smartphone nowadays with the help of accessibility tools present in them, we thought it would be a great tool to solve such problem. …


Image for post
Image for post

Everything starts with an idea. A year back I was a web developer and a designer. I am a curious fellow but I was uncertain whats next for me. Then I stumbled upon Artificial Intelligence, mostly through my facebook and twitter feeds. I had to know what it was all about, so I did. I took some courses on Machine Learning online, started practicing using python and it took me about a year or so to finally understand popular concepts and able to implement applications of deep learning using frameworks such as Keras and PyTorch. …


Image for post
Image for post

This is Lesson 7 of a series called Faster AI. If you haven’t read Lesson 0, Lesson 1, Lesson 2, Lesson 3, Lesson 4, Lesson 5 and Lesson 6 please go through them first.

Two third of this lesson is about architectures in convolutional neural network and techniques to improve such networks and remaining part is about different types of RNNs. For the sake of simplicity, I have divided this lesson into 3 parts:

  1. Resnet [Time: 0:2:20]
  2. Multi Output & Heat Map [Time: 0:43:00]
  3. Gated Recurrent Unit (GRU) [Time: 1:44:50]

1. Resnet

Resnet is another architecture in Convolutional Neural Network. This network utilizes something called resnet blocks and to picture these resnet blocks, lets look at this…


Image for post
Image for post

This is Lesson 6 of a series called Faster AI. If you haven’t read Lesson 0, Lesson 1, Lesson 2, Lesson 3, Lesson 4 and Lesson 5 please go through them first.

This lesson is all about Recurrent Neural Networks (RNNs). For the sake of simplicity this lesson is divided into 4 parts:

  1. Overview of Recurrent Neural Networks [Time: 15:33]
  2. Implementing simple RNN to predict a character [Time: 25:30]
  3. Architectures in RNNs [Time: 51:36]
  4. LSTM and more [Time: 1:05:55]

1. Overview of Recurrent Neural Networks


Image for post
Image for post

This is Lesson 5 of a series called Faster AI. If you haven’t read Lesson 0, Lesson 1, Lesson 2, Lesson 3 and Lesson 4 please go through them first.

This lesson is more conceptual and goes into lots of different ideas in deep learning.

As usual for the sake of simplicity and convenience, I have divided this lesson into 3 parts:

  1. Analyzing Collaborative Filtering results [Time: 00:10:00]
  2. Natural Language Processing [Time: 00:34:00]
  3. Recurrent Neural Networks [Time: 1:43:06]

1. Analyzing Collaborative Filtering results

In our previous lesson, we went on briefly about collaborative filtering and how it can used to make recommender systems. …


Image for post
Image for post

This is Lesson 4 of a series called Faster AI. If you haven’t read Lesson 0, Lesson 1, Lesson 2 and Lesson 3, please go through them first.

In this lesson, we are going to learn about different types of optimization algorithms, semi supervised learning and collaborative filtering.

As usual, for the sake of simplicity, I have divided this lesson into 4 parts:

  1. Gradient Descent Algorithms in Excel [Time: 11:28]
  2. State Farm Distracted Driver Detection Competition [Time: 56:57]
  3. Pseudo Labeling (Semi Supervised Learning) [Time: 1:23:45]
  4. Collaborative Filtering (Path to NLP) [Time: 1:36:01]

Before diving into these parts, Jeremy quickly goes through Convolutional Networks in Excel and there he shows how convolution works by explaining each part in detail using Excel Spreadsheets. …

About

Deep Learning Journal

Deep Learning Lessons

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store