How I make CarbonDescent — Ride your real bicycle in GTA V
I’m a programmer, gamer and road cyclist. I got myself quarantine like everyone and cannot ride my bike outside anymore. The best solution is Zwift virtual ride. I got myself a poor man Zwift setup from a year ago. It has a trainer, speed, cadence, and heart rate sensor. I have a good time on Zwift but the real road is far better. I’m missing the open-world where I can get lost and accidentally found a new place. Zwift cannot offer the open-world experience, cannot turn anywhere, no car, no traffic. So I was looking for a game that offers real-world bike ride experience.
The first game in my mind is GTA V. It got a huge map, a beautiful environment and most importantly, It has bicycles you can ride. I found the interesting AI project “Universe GTAV” that allows training a self-driving car in GTA V from years ago. I think I could reverse engineer how it controls the car and if the AI is really that good maybe I could make the autopilot option for Zwift like experience. It took me a couple of days to set up the Universe GTAV because the owner of the project got some legal issues with the game developer and the project source code and information got wiped out from history. I feel like I am an archeologist to resurrect the project. I got the pre-train model from the project called “deepdrive-universe” with some code fixing and the AI for a self-driving car in GTAV is up and running but It’s hardly usable for the in-game driving.
But the whole research is not going to be wasted. I learned how it controls the car by setting up the joystick emulator via vJoy software. vJoy offer a good library for controlling joystick from another program. The vJoy library has been ported to many languages including JavaScript(NodeJS), my goto full-stack development language. I wrote a simple program to control car steering in GTAV with the lib and it works flawlessly. More importantly, it does not require any mod to work (Hope Rockstar Games won’t sue me for this).
Now how to control the bike speed in the game. The state of bicycle acceleration in GTA V has two states. The normal pedal and sprint. Maybe I can try to control speed by press the button, un press for a while and press it again. This pressing step becomes a sprint. I don't know why the game developer did not let bicycle fine control speed like the car. Maybe they know someone gonna tries to hook the real bicycle into their game and Zwift will go bankrupt. It’s just my conspiracy theory. I simply fix this by pressing the button to pedal while user ride speed above a threshold. The threshold is 25km/h, easy for everyone to ride and let the bicycle speed in the game goes whatever it be.
Getting data from the bicycle sensors. The speed sensor is mandatory for this project set up and other data might be good for riding. I got my ANT+ USB stick from my Zwift set up so I will use it for this project to get speed data to press the pedal button in the game and these data from the sensors can be display on the mobile app like a bicycle computer unit. The data will be sent to the app by WebSocket.
The reason why this project needs a mobile application is bicycle steering. Because of the AI, the self-driving project is not working, the rider needs to fine control the bike themselves from the handlebar. Using a phone attached to the handlebar to control the bike direction sound promised. First I try to use the compass API to see when the rider heading and how much the angle changes when the handlebar turn. The problem is the compass API is too slow for this job. I fix the issue by using the accelerometer sensor. Calculation of the angle change from the sensor x y z position is not in my knowledge. I felt bad that I forgot basic math. It’s not possible — No, it’s necessary, come on TARS. I searched for opensource Android projects that use the sensor to control steering and I found one that controls an RC car. It even has a filter to normalize the angle value. I quickly port it to JavaScript on my react native project. It turns out accelerometer sensor needs more steering than I ever have to correct the direction. Using the direction from the magnetometer sensor is faster than the compass but needs the data calculation like an accelerometer sensor and with a little tune, it works very well.
Naming the project, the hardest part of the project. Naming things is my superpower. I just figured out by seeing what it is and what it does and use a different word. This project is for a “bicycle” “ride”. My bicycle made from carbon fiber and I’m still a carbon-based life form. Carbon is what it is. Riding fast is what it does and It goes really fast when descent. CarbonDescent! It could change a letter with the same sound as Swift to Zwift but KarbonDezent felt like trying too hard. I stick with CarbonDescent.
The full project setup looks like below.
The ride:
Source code:
https://github.com/artiya4u/CarbonDescent
https://github.com/artiya4u/CarbonDescentApp
