Living on the edge: Zero Knowledge Human-assisted Machine Learning with Git and Bitcoin Part 3
In this article, I will describe how I’ve addressed one of the challenges I encountered while performing custom upgrades to make my truck autonomous driving. In Part 1 and Part 2, I lay the foundation for my technology stack, which involves a human-assisted, reinforced learning method powered by bitcoin.
Now to be clear, I was initially only aiming to achieve Level 1 or Level 2 self-driving (given my truck has a manual transmission, full self-driving won’t be achieved). Reaching level 1 is totally within my capacity, and — with a few nifty hardware upgrades — level 2 can even be achieved. However, with the following solution, I may even be able to reach level 3!
Let’s build.
Our first task is to provide “eyes” to our truck. For starting this, we use a front-facing, single 1080p camera with a wide angle lens. This camera does a great job of capturing the road lines, signs, signals, and other objects. The eyes feed the real-time video stream to the “brain”, and the brain is then trained to recognize lanes and signs. Initially I was designing a model that would do the lane detection on the on-board computer brain, and that brain would then send me real-time feedback of my lane position, and eventually predict when we were “drifting” or about to leave the center of a lane. Well, with our new approach, we scrap this plan (mostly).
Computer Vision Machine Learning
Our solution involves offloading the “brain” tasks in a decentralized manner. For our video processing, we now use webRTC protocol to stream the realtime feed to a remote peer. WebRTC is an excellent choice due to is pure peer-to-peer nature, encryption by default, and ultra low latency. To ensure we securely connect to the peer machine of our choice, we use zero-knowledge on bitcoin as a part of our signaling mechanism. This approach also opens up the ability to do some interesting things such as offering a bounty to remote peers to compete for the best “autonomous driving service”. More on that later.
With our real-time stream now going to a remote peer, we can now pipe our video feed into our computer vision services. This remote peer can be a much more powerful machine that is, for example, in a secure data center, or running in our parents basement. With this remote feed, we also now introduce emotion recognition into the equation. Driving is certainly mostly a mechanical activity, but more often than not, we make split decision based on factors other than the parameters that we receive from the environment. For instance in the image above, we may decide to make that maneuver if after we have determined that the driver in front of us is slowing down or driving erratically, lets say. This is where a human-assisted reinforced learning model would serve us well.
Lane Detection and Real-time Feedback
Now, with the reduced need for a powerful brain on board the truck, I switched from a big computer setup down to a Jetson Nano. The camera feed is streamed to my remote machine, and I use GStreamer to pipe the video into the lane detection system, and output parameters such as lane count and lane position. These parameters are streamed instantly to the graph database, where simple algorithms run to determine if lil yota drifts to far left or right in the lane.
For testing and demonstration, I’ve developed a simple real-time feedback application on my ViiM Smart Glasses, that I can wear while driving. The application simply displays a blinking red or yellow signal on the left or right of the display based on my position in the lane. This, effectively is Level 1 autonomous driving.
Taking the fast lane with blockchain
The implementation we just described can be considered somewhat intermediate; we introduced a secure remote viewing system and real-time heads-up display. Well, lets think outside of the box for a moment. How can we use this setup to really accelerate the learning for our autonomous system? One way that comes to mind hearkens back on our need to build models with emotional intelligence. Lets say instead of my real-time stream going to just another computer, it also goes to an actual person or multiple people. Each of these people are then able to run their own unique computer vision models and provide real-time input to our information graph, as outlined in the diagram above. This could include providing labeled training data for our models, real-time alerts or warnings to the driver, or even go so far as complete remote control of the vehicle. Let’s build this out!
Real-time Peer-to-Peer Driving
I enhance my solution by making a few modifications to the plan mentioned above, without any major changes to our tech stack. Since we are using bitcoin as a part of our communication protocol stack, I can create a smart contract that offers cash rewards in exchange for real-time driving peers. Anyone (or computer) I choose can drive along with me in real-time. In an ideal world, these peers would compete to see who can provide the best methods and feedback in the form of training data, real-time features, or post ride analytics. To achieve this, the stream from my jetson is piped to Cloudflare WebRTC stream service, where I’ve programmed it to be accessed by specific viewers I choose. I can offer up the bounty to be paid upon completion of the ride. By having my AI model on bitcoin as well, I can add additional cash incentive for any training data submitted for training the model. This would save me the time of training the lane detection model, and I could get to the next steps required for Level 2 self driving orders of magnitude faster. The specific goal here, would be to implement a prediction tool that would accurately forecast when a lane drift may be about to occur, which is key to signaling the autonomous driving mechanics to make the necessary steering adjustments.
Summary
I’ve demonstrated a real-time zero knowledge autonomous vehicle model that provides predictive lane assistance. I was initially pretty bummed out when I fried my electrical in my truck trying to run a big powerful “brain” computer, but as it turns out, I was actually able to correct my issue with a bit of thinking “outside the box”. In doing so, I was able to devise an unbounded parallel computer stack that has benefits beyond just lane assistance. I can now add additional cameras and sensors to the system in a decentralized manner that does not require my strict attention to a single point of failure. Each of these new sensors can feed into multiple models, and with my bitcoin ML market place, I can find the best models that work for me. I will continue to build and share more as I go!
References
Machine Learning Marketplace on Bitcoin — https://xiaohuiliu.medium.com/machine-learning-marketplace-on-bitcoin-d8eb577be812
Modeling and Predicting Emotion in Music — http://www.met-lab.org/emotion-recognition
How to Train AI using Bitcoin — https://xiaohuiliu.medium.com/how-to-train-ai-using-bitcoin-3a20ef620143
AWARE by ViiM — https://www.viim.shop/about-us
