A Journey To Virtual Reality with React
Wow such buzzwords — but it works
I have this e̶x̶i̶s̶t̶e̶n̶t̶i̶a̶l̶ ̶c̶r̶i̶s̶i̶s worry that web development skills aren’t going to be quite as important in our future. There’s a lot brewing in the AI and Virtual Reality spaces, and well, it doesn’t look like what I work on day-to-day.
Over the holidays, I set out to “do some VR.” Here’s what happened.
A-Frame
A-Frame came out pretty recently, so it was fresh in my memory, and was my first foray.
Its novel idea is to use HTML to describe your world. This is a pretty low-friction way of creating a “static” scene — you can position primitive objects like cubes, texture them, and add animations all with markup. I can envision something like this logo animation in the header of a future website, and it’s not that big of a deal to code.
In addition to the markup, A-Frame detects if you’re using a VR-compatible device and seamlessly deals with the head-tracking and presentation changes.
Using HTML significantly lowers the barrier to virtual reality development, but eventually the declarative format falls apart once you want something interactive. A-Frame makes an effort to help with that (take a look at this interaction example), but I have experience shoehorning interactive logic into declarative formats and it rarely ends well.
An interesting side-effect of leveraging HTML is that you have all the DOM APIs at your disposal and are free to write interaction code in JavaScript. But now you’re back to the age-old problem of writing web applications in a way that doesn’t make our head explode. I thought to myself, “Didn’t we solve that already?”
A-Frame + React
I wasn’t the only person with that idea (cue the warm-fuzzies). AFrame-React is a pretty small library that lets you construct the A-Frame HTML elements with React.
It works, and let me have interactive logic with greater ease, but something bugged me.
You see, A-Frame is basically a markup abstraction on top of Three.js (which is itself an abstraction on top of WebGL). So by using AFrame-React, I was using JavaScript to control HTML elements that controlled…more JavaScript. A bunch of hoops.
An Aside: 3D DOM Elements
A-Frame (and most 3D web libraries) create shapes with textures like images and colors. But I found out that making those shapes textured with vanilla DOM elements is…interesting.
All 3D “stuff” in the browser happens within a canvas element, where the browser defers more low-level rendering capabilities to the application code. You can’t exactly take a DOM element rendered outside of a canvas and make it a first-class object within the canvas.
Yet there is a hack: you can synchronize the properties of a 3D object within the canvas to the CSS transform of a DOM element outside the canvas. This article gets linked around when you want to deal with this concept, but things become sort of hairy because of Three.js implementation details.
I did a bit of work into making React’s normal DOM elements and A-Frame work together. But the changes I had to introduce into A-Frame coupled with my eyebrow-furrow about jumping around JS-to-HTML-to-JS told me I should figure out something else.
React-VR
So I did that unfortunate thing where I rolled my own. If you 1) want VR capabilities 2) are okay with ThreeJS primitives 3) a React API, then you should check out my React-VR project.
It wasn’t that crazy to implement. Most of the heavy lifting is done by an existing React-ThreeJS library; all my project does is add support for the multi-camera setup required for VR (the code of which I nabbed from A-Frame).
It’s not on NPM and probably won’t be worked on very much in the future, but it was enough for me to feel a bit better about the future of VR (and myself) on the web.
I haven’t touched 3D or game development since I was a wee lad, and the fact that 3D rendering works is mad impressive to me now in a way I didn’t appreciate earlier.
I don’t know if VR will be a thing in popular culture, but it’s exciting to think about what even lowly web developers like me could make — whether using A-Frame, React, or proper tools like Unreal and Unity.
