TL;DR I built a VR setup for rodents from scratch and trained three rats in an automated fashion, without manual intervention, to traverse a corridor rendered in the DOOM II engine. Although I did implement the mechanisms to further train rats to shoot monsters in-game, I lacked the time to actually reinforce the behavior. The promise of the project is a relatively cheap (<$2000) VR setup that automatically trains rodents to traverse 3D environments without restraining them too much, while refraining from surgical procedures to provide the least stressful circumstances for them. Rodent VR rigs have had their presence in neuroscience experiments in the past; I hope to see more such studies to come! The following project aims to ease those experiments by automating some tasks of the experimenter.
This article describes the implementation of the automated training procedure proposed in my previous post: A Neuroengineer’s Guide on Training Rats to Play Doom
As a personal side project, building the rig took around 4 months and costed me more than two times of what the final product amounts to. Several rodent VR setup designs have been developed , however, most are difficult to get right or they require special equipment to assemble. As far as tools go, I had access to a medium-sized 3D printer, a soldering iron, a drilling machine and a screwdriver set. Wielding my Prime membership, I received 99 individual Amazon orders — special thanks to Yuniel for relentlessly delivering these packages to the lab!
Rodent VR setups usually consists of a (polystyrene) ball, on which the rat runs suspended in a harness, along with motion sensors tracking the movement of the ball. Either a screen or a projector displays the virtual environment. The ball could be levitated on airflow or just be rolled on ball bearings. I tried both options and the latter prevailed with its robustness despite the higher friction it imposed. I housed the whole system on an aluminum frame precut and ordered from Misumi.
Rats’ fields of vision extends to 300°; covering the whole field is not necessary [2, 3], but wrapping a wider screen around the animal ups the immersion factor. To simplify the setup, I avoided using projectors and went for a very curved commercial Dell screen that somewhat hugged the visual space.
Custom parts (attachments, items holding electrical components, mechanical contraptions) were 3D designed in Blender then printed. Dozens of versions were tested to arrive at the final rig.
A rubber-coated 16" (40 cm) foam ball was held in place by three ball bearings on the bottom, two at the front and one on the left side. The guiding principles for designing the base were as follows: 1) the ball bearings should contact the spherical treadmill at a 90° angle to minimize friction, 2) the arms should be long enough to support the treadmill from the sides preventing it from rolling off the base, 3) the base should be able to hold up foam balls of different sizes without manual redesign — at this point the diameter of the ball had not been fixed.
To meet all these requirements I wrote a Blender Python script that spawned the arms in the 3D editor just in the right position above the base, hugging the ball, given the dimensions of the foam ball, of the 3D printer and of the ball bearings. After changing the radius of the spherical treadmill in the script, a new arm arrangement can be spawned at the right size, angle and position. The arms were printed separately from the base to maximize their length.
The motion laser sensors were fashioned from gutted computer mice, and were installed at the front and left side to cover all three axes of ball movement. The motion detection range of the laser mouse maxed out around 1 cm. To account for irregularities on the surface of the foam ball, frontal bearings were positioned on magic arms distancing the ball from the front sensor. Magic arms further held the motion sensors and the reward stations, which made all this equipment easy to (re)position.
Rats were trained to move on the spherical treadmill by means of operant conditioning, which required a feedback mechanism that reinforces the desired behavior. To deliver positive feedback, I chose feeding them sugary water, as it’s relatively simple to dose in small amounts (tens of μL): took a precision solenoid valve, gravity-fed it (check the above intro video from 0:42) and drove it with a microcontroller. The above shown 3D printed tiny bowl functioned as a buffer for any excess water, while Quidditch goal posts held the tubing tight.
The two larger tubes conducted air puffs from the 12V pumps. Air puffs in the face or the butt are often used as negative feedback in rodent VR experiments ; specifically to signal when the animal hits the wall, from either side, while running. The foam padding were added to absorb the vibration of the pumps, which reduced noise. Reducing noise across the setup is in general of importance, as rodents can easily get startled by sounds of sudden onset. Although the air pumps worked fine, in the lack of time and out of caution, I refrained from applying air puffs in my experiments not to discourage rats from staying on the treadmill.
The equipment showed so far is part of most VR setups and is paramount for rodent VR experiments to work in the scenario of operant conditioning. The novel aspect of this project lies in the automatic training of rodents to traverse and perform actions in virtual spaces.
Train to Walk
Training rats to walk in VR is conceptually simple and has been done in a manual method: if the animal is stuck, roll the ball in the right direction, forcing the rat to take steps, then provide positive feedback to reinforce this behavior. When done manually, the experimenter has to roll the ball by hand before triggering the reward. Conversely, an automatic solution necessitates rolling the ball along two axes by the use of motors: forward-backward and left-right.
The video above first shows how the motor attempts (but fails) to rotate the ball: a linear actuator was extended until the wheel reached the ball, the motor rotated the wheel for two seconds before the actuator was detracted. Failing to roll the ball in this case was both a bug and a feature. If the rat put all its weight lying on the treadmill, the traction of the silicone wheel on the rubber-coated polystyrene ball turned out to be insufficient. The rat was not entirely forced to walk, only encouraged, which, I can anecdotally say, eased the acclimation process.
The second portion of the video demonstrates the 2D movement training. As you may notice, the rat was slightly confused about the second motor, which kept turning him to the right to face the exit sign — this was the first time Romero (that’s his name) ever encountered the enforced left-right rotation of the ball.
The last part of the video (from 0:32, turn the audio on) records a later part of the training, where Romero already learnt to take advantage of the motor and roll with it. He also just began to walk on his own there.
Placing a spring between the actuator and the wheel helped to maintain the traction of the wheel on the imperfect spherical surface — the distance from the center and the surface of the ball varied. A spring allowed for such variance without pushing the ball too much to the side.
Train to Shoot
Shooting is the in-game action that has to be performed in the context of a monster being in front of the player. Consistently training such a behavior is difficult manually, as monsters may appear in different spots and in large numbers. One could incrementally train a rat to shoot a single paralyzed monster, then a moving one, then multiple ones moving around, but again, to consistently induce the action associated with shooting, manual interventions would just not cut it.
Let’s say the behavioral action associated with shooting is the rat raising its body by extending the front legs (a form of rearing). It should be trained to perform this action every time a monster appears ahead. Every such time, the experimenter could manually lift the rat by the harness, initiate an in-game shot and release some sugary water for positive feedback; the timing of all these operations are essential to shorten the training period as much as possible, and to avoid confusing the animal by giving inconsistent feedback. So, how could this be automated?
A cute little harness (selected from a set of 5) was attached to a 3D printed arm allowing for vertical movement. A solenoid push-pull actuator, secured to the base of the arm, served as the training mechanism to teach the shooting behavior. Just above the actuator a button sat. When pushed, either by activating, pulling up the solenoid itself, or by the animal raising its posture and thus lifting the inactive solenoid shaft, an in-game shot was discharged. Note: the actuator was activated using pulse-width modulation to drive a more gradual pull not to scare the animal.
Simply put, the training procedure would go as follows: the rat walks into a monster → the software detects that the monster is in the proximity of the player (and for now, let’s assume that the player is facing it) → initially the rat has no idea what to do in this situation, so the training software activates the push-pull solenoid lifting the animal slightly upwards → the head of the actuator then touches the button → monster gets shot down → reward in the form of sugary water is released to reinforce the behavior.
Ideally, the rat learns on its own to rear up the body in front of every monster, which then lifts the shaft of the solenoid, pushes the button, the monster gets shot and the reward is dispensed.
Two Arduino microcontrollers managed all the electronics: 2 × (linear actuator + motor), 2 × air pump, a solenoid water valve, a push-pull solenoid, and the fire button. The motion sensors and the microcontrollers were wired to the PC via USB, each Arduino board communicating on a serial port, while motion data was received through the input event interface of Linux using the Python evdev library.
I wanted to push as much logic to the PC (Python) as possible without compromising on the response time of the setup. The PC sent small packages to the microcontrollers as commands: first byte being the message type (e.g. extend the linear actuator, dispense sugary water, etc.), then the packet load (how long to extend the actuator in seconds, amount of water to dispense in μL). The experiment described below was implemented in a single Python script (<300 lines), concurrently receiving movement and shot info, sending out commands to the devices and interfacing the game.
I designed a map containing a corridor, an exit door and an exit button in Doom Builder 2. The door and the button were activated on touch. Later to train the shooting behavior, I added an imp monster around 1/3rd of the corridor. The assets were taken from the first map of Doom II, Entryway. As I described in my previous post, this map enabled the first stage of training only requiring forward movement, and shooting a paralyzed, docile monster.
As the player pushed the button, the animal was teleported to the beginning of the map to start all over again. A Doom (ACS) script registered the player’s position along the corridor and the number of monsters it killed, and exposed all this info to the Python experiment script.
I ran Doom, accessed game states and sent game actions using the ViZDoom engine. ViZDoom was primarily built for training reinforcement learning algorithms, but its seamless Doom API made it perfect for the job. From motion data that evdev supplied, I calculated the speed of movement over time, scaled and relayed it to ViZDoom at every iteration of the game to translate rat steps into in-game steps. ViZDoom in return exposed the game state, sprinkled with some extra variables that were defined in the ACS script.
I received three 8-week-old male Long Evans rats, Romero, Carmack and Tom. Romero was fearless (more like thrill-seeking) and loved grapes. Carmack was a real architect building around its home keeping it tidy; he was fond of bananas. Tom began shy, but held the most surprises in learning performance.
I spent around 6 weeks with them during their (and my) night cycle, ~1 hour with each daily. It took two weeks to habituate them to me — this process could be accelerated, I was just a little slow with them. One week to teach them to stay put while I was dressing them; I also tested multiple harnesses and their combinations during this time.
I spend another week of habituation to the spherical treadmill and feeding from the bowl. First, I kept filling the bowl with baby food to make them stay on top of the ball. Baby food has a more compelling scent to recognize than sugary water; also, I could avoid instantiating water restriction for this period.
For the remaining 11 days I ran the actual VR Doom training. Romero outperformed the others in general, though Tom was the first to actually walk on its own without the help of the motorized ball roller. I had two sessions to train shooting with Romero, who was pretty confident running in VR at that point. Two sessions were far from enough; getting stuck into the monster and pulled up by the solenoid confused him. In the last part of the video below, you can hear the push-pull solenoid activating, and see it pulling Romero slightly upwards before the shots are discharged.
Amid the VR training I kept improving the software simultaneously with the hardware, which inconsistency most probably interfered with the learning progress, but was necessary to finalize the setup.
At this point you might rightfully ask, why? Playing games, whether a human does it or rats, involves the interplay of a wide variety of cognitive processes. A rat, being stationary relative to the setup, can express a wide variety of behavior, while also being recorded by a neural interface. Thus, movement and complex in-game actions can be correlated with neural activity in a virtual, yet more natural, lively set of experiments, than in simple maze setups for instance.
Computer games, virtual worlds are abundant and easy to make. Once we can reliably train animals to play games, designing an experiment becomes a software problem instead of an often costly hardware problem. However, consistent training of animals to perform complex, context-dependent actions in VR happens to be pretty difficult. This project was an attempt to automate this process. However, even after 11 days of VR training, I had to be present and help the rats to get on or off the ball, as they desired. Although their training was software-based, it still required my presence.
Unfortunately, I did not have the time to carry out the experiment in its entirety — I gave myself a hard deadline out of personal reasons. Rats lacking the ability to shoot is an obvious major limitation of this study. In hindsight, I would much rather implement an already tested response method to train shooting, like a nose-poking system, than the button-pushing contraption. Although the mechanisms to train left-right turning was present, I did not have time to train movements with turns included, so in most sessions only one axis of motion was registered.
XI had fun building a rodent VR rig and training rats to kinda play Doom. Apart from paying for the animals, their housing and access to lab space, I financed the project myself; I wanted to see it through and worked most nights building, coding and conducting experiments. I left out a great deal of details from this article: rubber-coating 4 huge foam balls, one with a 24" diameter, testing various coating materials; levitating the treadmill with air pumps implementing a design  that I then had to throw out entirely; several attempts at recalibrating the motion sensing code, so rat movement got translated into immediate in-game steps of the right speed and consistency; and many more. Honestly, I thought it would be way easier, especially the training part, but one can’t underestimate the often unyielding independence of these free spirited animals.
The above presented VR system is ridiculously cheap to build at a price below $2000, compared to the other commercially available option costing north of $70,000. There’s an undeniable difference in product quality, and the scope of testing, but not as high to justify this contrast in price. I hope to see more preclinical neuroscience research capitalize on VR experiments to capture behaviorally and cognitively rich neural recordings.
Hit me up at firstname.lastname@example.org if you have any questions! To see more on the why, check out my previous article: A Neuroengineer’s Guide on Training Rats to Play Doom
I would like to say thanks to a bunch of friends, colleagues, who I could reach out to for professional assistance. Without them this project would have halted in the very beginning.
Thank you Fylaktis, Jose, Rich, Santosh and the two Jasons for giving me support on electrical and mechanical issues! Yuniel, thank you for delivering all those packages! Thanks Christine for patching up the rat harness and all the encouragement on the way! Umair and Franka thank you for helping me to receive animals and for providing guidance in handling them! A very special thanks to Ibrahim who provided me constant assistance in designing the setup and the experiments, in addition to encouraging me on the way! Naveen, thanks a lot for supporting the project throughout and giving the idea to attach springs on the linear actuators! Thanks Todd for sharing my love for Doom and encouraging me! Thank you Pedro for carrying and making space for this huge setup! Special thanks to Theo, Pato and their lab members for providing me with lab space and putting up with my crazy idea!
Romero, Carmack and Tom, both in rat and human forms, thank you for making this possible!
Rats Play Doom / MAP01
The first RPD T in limited black and redJust take a look at your left sleeve and don't miss the secrets!Black Ts are…
 K. Thurley and A. Ayaz, “Virtual reality systems for rodents,” Curr Zool, vol. 63, no. 1, pp. 109–119, Feb. 2017, doi: 10.1093/cz/zow070. B. K. Young, J. N. Brennan, P. Wang, and N. Tian, “Virtual reality method to analyze visual recognition in mice,” PLoS One, vol. 13, no. 5, p. e0196563, 2018, doi: 10.1371/journal.pone.0196563. G. Chen, J. A. King, N. Burgess, and J. O’Keefe, “How vision and movement combine in the hippocampal place code,” PNAS, vol. 110, no. 1, pp. 378–383, Jan. 2013, doi: 10.1073/pnas.1215834110. T. Muzzu, S. Mitolo, G. P. Gava, and S. R. Schultz, “Encoding of locomotion kinematics in the mouse cerebellum,” PLOS ONE, vol. 13, no. 9, p. e0203900, Sep. 2018, doi: 10.1371/journal.pone.0203900. D. B. Aharoni, “Rats in Virtual Space: The development and implementation of a multimodal virtual reality system for small animals,” UCLA, 2013. Accessed: May 03, 2021. [Online]. Available: https://escholarship.org/uc/item/0wd4p4mr
The experiment was approved by the Feinstein Institutes’ Institutional Animal Care and Use Committee.