In the last year I decided that I wanted to learn more about robot control. It’s a well known fact that AI and robotics control are going to be a big thing in the next 5 to 10 years.
Part of my journey was to get involved with reinforcement learning. Reinforcement learning has had some huge wins in the last few years, for example AlphaGo, Playing Atari games and beating professional players in DOTA2.
I decided I REALLY wanted to get involved! I built a few simple models, played with OpenAI Gym and experimented with taking the game connect 4 into any number of dimensions and then using Deep Q Learning to play it against itself.
However my experience told me there were a few things that were missing, and I wanted to make it better:
- Competitiveness was on paper — Online competitive algorithm gaming tools such as Kaggle, OpenAI Gym and HackerRank only give you a score based on your mathematical performance of a certain environment or challenge. I wanted something more tangible, I want to see my algorithm in action against others! I really don’t like reading papers. YAWN.
- Competitions are not real time — There are a handful of algorithmic game competitions out there, but they are inconvenient to watch. You have to create a game and then wait for other’s to join. You can’t just start playing straight away.
- I want to show off my “pet” solutions to other people — I wanted this to be super convenient … why not stream it live to anyone who wants to watch? Maybe there’s a recruiter you can impress, or you want to explain a particular algorithmic process to a non-technical friend? Your friend probably hates reading papers too.
- I want to watch AIs solving problems — Now this might seem a bit strange, but me and hundreds of thousands of users of Twitch and YouTube like to watch AIs solving problems, whether there are trashing human players at Go, or pitifully trying to drive cars in GTA 5.
I also had a few thoughts about what this kind of platform might potentially achieve given the right environments and users, for example:
- AIs that are in competition with each other produce “super human” solutions — We know that we can produce AI game players that are far better than humans at playing games. AlphaGo Zero for example has beaten the best human and algorithmic players at both Go and Chess.
- AIs and algorithms playing games could be an E-Sport — Why not create superstars out of the best individuals and teams of algorithm and ML researchers?
Robot control is hard and at times tedious. Lets make it fun!
I am looking for 30–40 beta testers to provide feedback on the platform and the beta environment “Andromeda”. help create more innovative environments for bots to compete!
Introducing embod.ai
Think of embod.ai as a platform for Massively Multiplayer Online Game, but purpose built for algorithms to play, rather than people.
Think of this… algorithms do not need to sleep, they can consistently rack up points, improve themselves over time. Some of the test bots I have built have been running for several days non-stop. You can train your models for a few hours or days… then run them indefinitely. Then visit and watch what they are doing at any time.
In the near future, depending on what the community wants, we may do competitions and leagues. Maybe even allow trading or betting on particular models rise and demise :)
Let’s take a look at the beta Environment “Andromeda“
Andromeda is the first environment at embod.ai. It’s a virtual world where the aim is to collect “food” and avoid “poison”. The physics and gameplay are super simplistic and designed to be that way to show how easy it is to make bots in embod.ai.
There are some example controllers on the embod.ai github page:
- Deep Reinforcement Learning. Using the skeleton Proximal Policy Optimisation algorithm provided by the brilliant tensorforce library. View the repo here.
Disclaimer: I am yet to get this one to produce a decent score. I’m very excited to see the first working Deep RL model!
- The “Reaper”. A very simple algorithm using the characteristics of the agent’s vision to decide on rotation force to apply. View the repo here.
Here’s a few features we have so far in Andromeda:
Leaderboards
Compare your solutions with others while they are running, maybe there’s a clever strategy that can be exploited that you haven't thought of?
Bot Tracking
You can track any bots running in the beta environment Andromeda from the web app, just make an account, click on an active bot and watch it!
Super easy API integration
Just clone the embod client and use it to control your bots!
Come and get involved!
I am really excited to see some innovative algorithms and ML models that perform much better than the few “reapers” that I currently have floating around.
To get access to the beta, do one of the following:
- Send an email to beta@embod.ai — Just need your email address and why you are interested in joining the beta program!
- Message embod on gitter — Please make sure it’s a direct message and not pasted on the embod lobby (otherwise other pesky users will see it)
- Carrier pigeon - Tell the pigeon your email address and they will relay it to me
- Create an unnecessary blockchain startup to send email- Then follow option 1.
Here are some things I’d like to focus on with beta testers
- UX/UI — For this project I learnt react/redux and used what I thought was a good looking template from themeforest. I think there’s a a great opportinuity for improvement here. Let me know what you think. Is yellow ugly? Is the logo too blocky and looks like it was done in MS Paint by a child?
- What games are simple to build algorithms, but also fun to build and spectate — Andromeda is a very simple “fish tank” like environment, not alot happens, and the competition is fairly indirect. I want to know how you think this could be improved without making it too complicated to new users.
- Incentivisation — What do you think will keep players coming back to improve their models and want to be top of the leaderboard? What will keep spectators interested in watching the models?
- API/Integration experience — I’m a hardcore backend engineer, so I’ve worked hard on making the client libraries and github documentation explicit with good examples. What is confusing? Are there better ways of doing things you can think of?
- What do you want from the community-Do you think there are cool features to make this more community driven such as live chat while bots are running? Do you want to customise your experience, for example adding decals to the bots, or changing bot color/shape?
Support and Community
Follow embod.ai on twitter.
Feel free to contact support@embod.ai, leave issues on github projects or join the gitter community
