A Journey To Virtual Reality with React
I have this e̶x̶i̶s̶t̶e̶n̶t̶i̶a̶l̶ ̶c̶r̶i̶s̶i̶s worry that web development skills aren’t going to be quite as important in our future. There’s a lot brewing in the AI and Virtual Reality spaces, and well, it doesn’t look like what I work on day-to-day.
Over the holidays, I set out to “do some VR.” Here’s what happened.
A-Frame came out pretty recently, so it was fresh in my memory, and was my first foray.
Its novel idea is to use HTML to describe your world. This is a pretty low-friction way of creating a “static” scene — you can position primitive objects like cubes, texture them, and add animations all with markup. I can envision something like this logo animation in the header of a future website, and it’s not that big of a deal to code.
In addition to the markup, A-Frame detects if you’re using a VR-compatible device and seamlessly deals with the head-tracking and presentation changes.
Using HTML significantly lowers the barrier to virtual reality development, but eventually the declarative format falls apart once you want something interactive. A-Frame makes an effort to help with that (take a look at this interaction example), but I have experience shoehorning interactive logic into declarative formats and it rarely ends well.
A-Frame + React
I wasn’t the only person with that idea (cue the warm-fuzzies). AFrame-React is a pretty small library that lets you construct the A-Frame HTML elements with React.
It works, and let me have interactive logic with greater ease, but something bugged me.
An Aside: 3D DOM Elements
A-Frame (and most 3D web libraries) create shapes with textures like images and colors. But I found out that making those shapes textured with vanilla DOM elements is…interesting.
All 3D “stuff” in the browser happens within a canvas element, where the browser defers more low-level rendering capabilities to the application code. You can’t exactly take a DOM element rendered outside of a canvas and make it a first-class object within the canvas.
Yet there is a hack: you can synchronize the properties of a 3D object within the canvas to the CSS transform of a DOM element outside the canvas. This article gets linked around when you want to deal with this concept, but things become sort of hairy because of Three.js implementation details.
I did a bit of work into making React’s normal DOM elements and A-Frame work together. But the changes I had to introduce into A-Frame coupled with my eyebrow-furrow about jumping around JS-to-HTML-to-JS told me I should figure out something else.
So I did that unfortunate thing where I rolled my own. If you 1) want VR capabilities 2) are okay with ThreeJS primitives 3) a React API, then you should check out my React-VR project.
It wasn’t that crazy to implement. Most of the heavy lifting is done by an existing React-ThreeJS library; all my project does is add support for the multi-camera setup required for VR (the code of which I nabbed from A-Frame).
It’s not on NPM and probably won’t be worked on very much in the future, but it was enough for me to feel a bit better about the future of VR (and myself) on the web.
I haven’t touched 3D or game development since I was a wee lad, and the fact that 3D rendering works is mad impressive to me now in a way I didn’t appreciate earlier.
I don’t know if VR will be a thing in popular culture, but it’s exciting to think about what even lowly web developers like me could make — whether using A-Frame, React, or proper tools like Unreal and Unity.