Feature thumbnail of the post containing, dog, cat, bone and robot icon and game screenshot
Feature thumbnail of the post containing, dog, cat, bone and robot icon and game screenshot

In my last post, I went through how we can create a custom maze adventure like environment using TF-Agents and how we can train, evaluate and visualize the performance of a DQN agent. In this post, I have added few more actions that our agent can perform and we will see how we can include that in the existing environment. …


Actions taken by agent/dog during 1 episode of gameplay and corresponding rewards and penalties received on each action step.
Actions taken by agent/dog during 1 episode of gameplay and corresponding rewards and penalties received on each action step.
Actions taken by agent/dog during 1 episode of gameplay and corresponding rewards and penalties received on each action step.

Deep Reinforcement Learning is a type of Reinforcement Learning algorithm which uses Deep Neural Networks that helps agents in making decisions. TF-Agents is a framework and a part of TensorFlow ecosystem that allows us to create such Deep Reinforcement Learning algorithms easily. In this blog I will explore basic of Deep Reinforcement Learning using TensorFlow Agents or TF-Agents in short by creating a simple custom game to illustrate the concepts and steps that goes in creating it using the framework.

Overview of the game

Reinforcement Learning is used in scenario where the system is required to make decisions based on the input they receive. To better understand this concept, let’s take an example of a custom game that we will create and let’s call it ‘Dog in a park’. The game is very simple, we have a dog who is the main player of this game and his name is Kiko. There are few challenges ahead for Kiko. He is in a section of a park where he isn’t supposed to be. But our Kiko is a very greedy dog, and he got there by following the smell of the bones lying on the ground. Every second he spend there is very risky and he needs to get out of that section as soon as possible. To make matter worst, there are already some robots present in that section to secure the area and if they see him, he is definitely in trouble. Now, the challenge for Kiko is, he needs to eat as much bones as he could find and as quickly as possible before reaching the exit and that too without being noticed by any of the robots. …


Image for post
Image for post
Fig 1: Image extracted from the original paper: https://arxiv.org/abs/1902.04502

Fast Segmentation Convolutional Neural Network (Fast-SCNN) is an above real-time semantic segmentation model on high resolution image data suited to efficient computation on embedded devices with low memory. The authors of the original paper are: Rudra PK Poudel, Stephan Liwicki and Roberto Cipolla. The code used in this article is not the official implementation from the authors but an attempt by me to re-construct the model as described on the paper.

Since the rise of autonomous vehicles, it is highly desirable that there exist a model that can process input in real-time. There already exist some state-of-the-art offline semantic segmentation models, but these models are large in size and memory requirement and requires expensive computation, Fast-SCNN can provide solution to all these problems. …


Image for post
Image for post

Since its inception, AID has conducted numerous workshops, meetup events, professional trainings in AI all over Nepal. We have also released prototype of our open collaboration project, Cash Recognition for Visually Impaired, which will enable visually impaired individuals in Nepal to do daily monetary tasks independently and confidently. The aim of these events and projects is to educate individuals who are new to AI and help them understand how they can get started with it and how they can use it to build solutions to their community. Soon after incorporating our community into AID, we partnered with CityAI, a global non-profit initiative and a community of AI practitioners across 40+ cities all over the world that shares challenges and lessons learned in applied AI. …


Image for post
Image for post

This article was first published on Deep Learning Journal

Recently we initiated Artificial Intelligence for Development or AID in short, to empower AI movement in Nepal. The idea is to develop AI talent in Nepal and connect them with domain experts from diverse fields such as health, education, agriculture, finance, transportation etc and create an environment to develop AI solutions to existing community problems in such fields.

This approach will develop AI ecosystem in Nepal and will address following issues:

  1. It will allow new AI learners to learn from practical problems in local community.
  2. AI developers will exercise their skills to develop solutions that are directly beneficial to the…


Image for post
Image for post

Recently we initiated Artificial Intelligence for Development or AID in short, to empower AI movement in Nepal. The idea is to develop AI talent in Nepal and connect them with domain experts from diverse fields such as health, education, agriculture, finance, transportation etc and create an environment to develop AI solutions to existing community problems in such fields.

This approach will develop AI ecosystem in Nepal and will address following issues:

  1. It will allow new AI learners to learn from practical problems in local community.
  2. AI developers will exercise their skills to develop solutions that are directly beneficial to the…


Couple of months ago, we, Developer Sessions team started a project for the blind community of Nepal. The project is Cash (Monetary Notes) recognition for visually impaired.

Here is Intel Software Blog article written by me about this project

In Nepal, the monetary notes (Cash) are not accessible for visually impaired individuals, there are no special markings in these notes to let them know what they are carrying. Some, with years of experience finally learned to recognize them while some still have to seek help of others to know the value of the note they are carrying.

We tried to solve this problem by creating a smartphone app. As most of the blind individuals uses smartphone nowadays with the help of accessibility tools present in them, we thought it would be a great tool to solve such problem. …


Image for post
Image for post

Everything starts with an idea. A year back I was a web developer and a designer. I am a curious fellow but I was uncertain whats next for me. Then I stumbled upon Artificial Intelligence, mostly through my facebook and twitter feeds. I had to know what it was all about, so I did. I took some courses on Machine Learning online, started practicing using python and it took me about a year or so to finally understand popular concepts and able to implement applications of deep learning using frameworks such as Keras and PyTorch. …


Image for post
Image for post

This is Lesson 7 of a series called Faster AI. If you haven’t read Lesson 0, Lesson 1, Lesson 2, Lesson 3, Lesson 4, Lesson 5 and Lesson 6 please go through them first.

Two third of this lesson is about architectures in convolutional neural network and techniques to improve such networks and remaining part is about different types of RNNs. For the sake of simplicity, I have divided this lesson into 3 parts:

  1. Resnet [Time: 0:2:20]
  2. Multi Output & Heat Map [Time: 0:43:00]
  3. Gated Recurrent Unit (GRU) [Time: 1:44:50]

1. Resnet

Resnet is another architecture in Convolutional Neural Network. This network utilizes something called resnet blocks and to picture these resnet blocks, lets look at this…


Image for post
Image for post

This is Lesson 6 of a series called Faster AI. If you haven’t read Lesson 0, Lesson 1, Lesson 2, Lesson 3, Lesson 4 and Lesson 5 please go through them first.

This lesson is all about Recurrent Neural Networks (RNNs). For the sake of simplicity this lesson is divided into 4 parts:

  1. Overview of Recurrent Neural Networks [Time: 15:33]
  2. Implementing simple RNN to predict a character [Time: 25:30]
  3. Architectures in RNNs [Time: 51:36]
  4. LSTM and more [Time: 1:05:55]

1. Overview of Recurrent Neural Networks

About

Kshitiz Rimal

AI Developer, Google Developers Expert (GDE) on ML, Intel AI Student Ambassador, Co-founder @ AI for Development: ainepal.org, City AI Ambassador: Kathmandu

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store