Behind the Scenes: The Video Games Team
Podcast & Transcript
Mahdi: Hello and welcome to Behind the Scenes with QMIND Design teams, I’m Mahdi Kabir from Educate, and I’m here today with Ted Munn, team lead of the Video Games team. Ted, can you tell me a little bit about your project and your team members?
Ted: Yeah! So our project is training an agent to play Sonic the Hedgehog using reinforcement learning. On my team I have Chen Sun, Jessica Li, Liam Gotye, Simon Dudtschak, and Vanessa Noel. The significance of this project is kind of derived from the goal of OpenAI, which is Elon Musk’s artificial intelligence company. Their goal is to try to achieve general intelligence using machine learning. What’s really interesting about this project is they chose video games — like their method of approach for this area is using video games — because video games are a very controllable control state space and it’s a really good environment to teach agents to learn independently.
Mahdi: Can you describe your project in terms of libraries and what other tools you’ve used?
Ted: Yeah so we started out this project using the OpenAI Gym Retro library. That library wraps around video game emulators. One of the projects or — one of the competitions that they ran first was Sonic, so we decided [that] because there’s a lot of good resources out there, to use that as our environment. So you buy the Sonic ROMs and then you wrap the slider around it. What it does is it provides input and output for your neural net. The input is just the pixel values of what’s been seen on screen. And the output is like the state action space (all the possible moves that the agent can take). What we do is we kind of feed these into either end of our neural net — what that allows for is the agent to see the screen just as a gamer would and then to interact with it and learn how to behave based on different awards that you give it. Beyond the video — beyond the Gym Retro library, we’re also using OpenCVs to convert the pixel values to grayscale to make it simpler for our model. And they were using Keras-RL which uses Tensorflow Backend for the actual neural net.
Mahdi: What do you think is the most challenging aspect of your project?
Ted: The challenges come in different iterations, right, so for us, the really big challenge was just getting the environments set up. Like it’s a really complex system and even getting to the point where we can be interacting with the system required a lot of setup. So that was kind of our version of data acquisition and once we got that right, we’re still running into challenges on how to train the agent, tuning the reward function, which is how the agent gets feedback for the actions that it takes, is also a big challenge. So I’d say there’s no specific one.
Mahdi: Speaking of your team, is it easy to coordinate times to work on your project?
Ted: So we have a large team — we might be one of the bigger ones. It’s me and five others actually on this project. So coordinating has definitely been a challenge. I’d say if it weren’t for the fact that we have days off on the weekend… What’s been nice though, is everyone’s pretty caught up and on the same page. So it’s really easy to communicate on Slack.
Mahdi: Spending so much time with your team, do you guys ever clash on ideas?
Ted: I wouldn’t say so. Like, because we have such a big team and because of the nature of how development kind of goes with this kind of project, what I’ve done is I’ve given a really good baseline of code to work on. And then what I’ve said is everyone can individually come up with their own machine learning models. We started out researching and right now we’re implementing them. So it’s not really — we don’t have to make choices about which direction to go. I came in with an idea of how we were going to do this and everyone’s just keen to learn about how this project is going to run, so there’s not really any conflict about who needs to do what, everyone just gets to do what they like.
Mahdi: I’m glad it worked out so well for you!
Ted: Yeah, it’s been great.
Mahdi: Thank you so much for sharing your project with me and —
Ted: No problem!
Mahdi: I’m sorry your team members couldn’t be here, but I wish you guys the best of luck!
Ted: Thank you!
