A long history with React on Web and Native led to a very natural exploration in the Virtual space. We would love to tell you about it!
At Airbnb, we’ve been using React to build user interfaces for our website since 2014. React has radically changed our approach to building UI, and we’ve been active contributors to the open source community with projects such as Enzyme, Hypernova, react-sketchapp, react-with-styles, react-dates, and react-native-maps.
React VR provided a unique opportunity to leverage our expertise to rapidly test and iterate many of the VR ideas we’ve been playing with. To do this, we brought together a small team of React and VR experts at Airbnb Samara’s VR Lab to see what we could accomplish in the 3 weeks leading up to the React VR launch at F8. Here’s an overview of the technology that powers React VR and the similarities between React, React Native, and React VR.
Why React
One of React’s biggest innovations is that it enables developers to describe a system, such as the UI of a web or mobile app, as a set of declarative components. The power of this declarative approach is that the description of the UI is decoupled from its implementation, allowing authors to build custom “renderers” that target more platforms than just web browsers, such as hardware, terminal applications, music synthesizers, and Sketch.app.
React as a paradigm is perfect for wrapping the complexity of underlying platform APIs and providing consistent and smooth tools to the developers using them.
Jon Gold, “Painting with Code” http://airbnb.design/painting-with-code
React Native Under the Hood
Because React VR is built on top of React Native, let’s start with a look at how it works under the hood. React Native is built on a renderer that controls native UI on iOS and Android. The React application code runs in a JavaScript virtual machine in a background thread on the mobile device, leaving the main thread free to render the native UI. React Native provides a bridge for communication between the native layer and the JavaScript layer of the app. When the React components in your application are rendered, the React Native renderer serializes all UI changes that need to happen into a JSON-based format and sends this payload asynchronously across the bridge. The native layer receives and deserializes this payload, updating the native UI accordingly.
Over the past year at Airbnb, we’ve invested heavily in React Native because we recognize the power of being able to share knowledge, engineers, and code across platforms. In November, we launched our new Experiences platform, which is largely written in React Native on our iOS and Android apps, and we formed a full-time React Native Infrastructure team to continue this investment.
Learn once, write anywhere.
Tom Occhino, React Native: Bringing modern web techniques to mobile
React VR is Built on React Native
React VR’s architecture mirrors that of React Native, with the React application code running in a background thread — in this case, a Web Worker in the web browser. When the application’s React components are rendered, React VR utilizes the React Native bridge to serialize any necessary UI changes and pass them to the main thread, which in this case is the browser’s main JavaScript runtime. Here, React VR utilizes a library from Oculus called OVRUI to translate the payload of pending UI updates into Three.js commands, rendering a 3D scene using WebGL.
Finally, React VR utilizes WebVR’s new navigator.getVRDisplays() API to send the 3D scene to the user’s head mounted display, such as an Oculus Rift, HTC Vive or Samsung Gear VR. WebVR, a new standard being spearheaded by Mozilla, is supported in recent builds of major web browsers. Check out webvr.info for the latest information on browser support.
Because React VR implements a lot of the same public APIs that React Native implements, we have access to the same vast ecosystem of patterns, libraries, and tools. It will feel familiar for any developer who has built React or React Native apps. We were able to get a VR prototype up and running quickly; in no time at all, we scaffolded a basic React application, set up Redux, and began hitting our production JSON API for sample data.
With hot module reloading and Chrome Dev tools debugging, we could iterate nearly as fast as in React web and React Native development, which allowed us to throw a bunch of UI ideas at the proverbial wall to see what would stick.
Using React (JavaScript) has turned out to be a bigger win for VR app development than I expected — UI dev is several x faster than Unity.
John Carmack, Oculus CTO and original creator of Quake.
Flexbox in VR
React VR inherits React Native’s flexbox-based layout and styling engine, with a few tweaks to allow transforms in 3 dimensions.
Flexbox support for React Native is provided by Yoga, a cross-platform layout engine created by Facebook to simplify mobile development by translating flexbox directives into layout measurements. Because it’s written in C, Yoga (née css-layout) can be embedded natively in Objective-C and Java mobile apps. React VR also uses Yoga for flexbox layout. “But how?” you ask, “It’s written in C!” The React VR team has accomplished this by using Emscripten to cross-compile the Yoga C code into JavaScript. How cool is that?
This is a powerful feature of React VR: developers can use the same styling and layout system across web, React Native, and VR, which opens the doors to directly sharing layout styles across these platforms.
Sharing Primitives
Like React Native, React VR provides a set of basic primitives used to construct UI—<View>, <Text>, <Image>, Stylesheet—in addition to adding of its own VR-specific primitives, such as <Pano>, <Box>, among others. This allowed us to drop in some of our existing React Native components into VR, rendered on a 2D surface.
This is hugely exciting because we’ve built our UI component system upon react-primitives, a library we developed for sharing React components across platforms by providing the basic <View>, <Text>, <Image>, etc. primitives for a variety of platforms, including web, native, and Sketch.app (via our react-sketchapp project).
This means we can use the buttons, rows, icons and more directly in VR, keeping Airbnb design language consistent without having to rewrite it all from scratch.
Check out our engineer Leland Richardson’s talk at React Europe for a more in-depth look at the promise of react-primitives, below.
<CylindricalLayer />
As you could imagine, placing 2D content onto a flat plane in 3D space often falls short of an optimal viewing experience. Currently, many VR apps solve this by rendering 2D UI onto a cylindrical plane curved in front of the viewer, giving it a “2.5D” feel.
React VR provides the <CylindricalLayer> component for this, which we found to be effective for displaying menus and other interactive content in our app, and was the perfect way to integrate the existing 2D React components from our UI library into the VR environment.
With the ability to leverage our existing React knowledge, tools, and UI component system, we were able to be productive in React VR almost immediately. Couple that with the magic of hot module reloading, which allowed us to tweak the orientation of objects in 3D space and receive instant feedback from someone immersed in the VR environment, we managed to churn out a large number of low-fidelity prototypes, test them with users, and quickly learn which of our ideas worked in VR and which didn’t.
And this kind of broad experimentation is necessary for folks first diving into VR development—we found time after time that a UX idea that sounds good on paper may very well feel strange once the VR headset is on, and many of the UI metaphors that work well in 2D don’t translate well to 3D. One of main takeaways from our 3 week sprint was that we have to start developing a new understanding of UX best practices for VR, and we can only fall back on our existing web and mobile learnings to a limited extent. Luckily, there are researchers forging these new paths, like the folks behind UX of VR, and we can learn from their hard work.
It’s also important to note that React VR, WebVR, and VR hardware are nascent technologies. The instability inherent in working on the cutting edge can make the development process frustrating and bumpy. On the flip side, it’s incredibly exciting how quickly the ecosystem is progressing, and React VR in particular has matured quickly in the three months since we used the pre-release version for our prototyping.
Lastly we’d like to thank the Facebook and Oculus teams who supported us during this exploration and we are excited to contribute in helping move the platform forward. We can’t wait to see what’s in store!
Prototyping with React VR was originally published in Airbnb Engineering & Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.
