Learn React VR (Chapter 3 | Outdoor Movie Theater)

Prerequisites

Chapter 1 | Hello Virtual World
Chapter 2 | Panoramic Road Trip

Scope of This Chapter

In the previous chapter, we were able to create a mini VR app called Panoramic Road Trip that allowed us to test the major features of React and see what was different when working with React VR. Overall, this was effective for learning but a bit simplistic in comparison to what we are capable of achieving with React VR.

In this chapter, we are going to have some fun making another VR app called Outdoor Movie Theater. As we make this app, we are going to learn how to work with sound and video in React VR. Additionally, we are going to look into how to better organize our project.

Creating a New Project

Let’s create a new project called Outdoor Movie Theater.

Change directories in command line to where you want the new project folder created.

Then, run the following:

react-vr init OutdoorMovieTheater

Once that is finished, add the project folder to Atom and open index.vr.js.

Organizing Our Project

Before we write code, it’s important to address how to better organize our React VR projects as we will be adding more complexity.

In order to organize our project, we are going to look into organizing our project directory and implementing a module system.

Organizing Our Project Directory

So far, we have just been writing all of our React components in the index.vr.js file. While this is fine for a simple app, we ideally want to divide our components into separate folders and files.

How you organize a React VR project’s components is completely personal preference. It also depends on the complexity of the app.

For I smaller-scale project like ours, I organize components into the following folders:

+ [Project Root]
+ Components
+ Scenes
+ Layouts
+ Elements

Within these folders, there will be files that each define a single React component.

Let’s go through mockups of our Outdoor Movie Theater app to understand why I prefer to organize my components in this fashion.

In this app, we are going to have various scenes.

For example, we will have a Main Menu scene:

We will also have a Theater scene for watching the outdoor movie (and some others as well):

Each scene is composed of elements which are contained in a flexbox layout.

Going back to the Main Menu scene, this scene is composed of 2 elements (a Title and Button) that are within a layout:

Therefore, we will have scenes, layouts, and elements within our VR app. Each of these can be broken down into their own components and organized hierarchically.

The elements components will be nested into a layout component which will then be nested into a scene component. Each of these will be separate folders and files to help organize and keep track of this hierarchy.

We will then have an app component in our index.vr.js which will contain the panoramic photo and multiple scenes. It will always be named after our project. We can use conditional rendering to control which scene displays.

The app component is the top of our component hierarchy which will ultimately get injected into our index.html file and then rendered to the browser.

If this all seems a bit fuzzy when trying to visualize, don’t fret! We are know going to create our Main Menu scene and make note of the application of what we just discussed.

Before we do that, let’s add our component folders to our project:

+ [Project Root]
+ Components
+ Scenes
+ Layouts
+ Elements

Implementing a Module System

If you are not familiar with a module system, in short, a module system allows us to write code to make the components exportable so that they can be imported elsewhere (i.e. making an element component exportable so it can be imported and nested in a layout component).

In other words, we can define our components in separate files and then use them elsewhere. We will write our module system using ES6.

Let’s start by updating the shell of our app component in index.vr.js:

export default class OutdoorMovieTheater extends React.Component {
render() {
return (
<View>
<Pano source={asset('fort-night.jpg')}/>
</View>
);
}
};

Here’s the link to download the fort-night.jpg. Make sure to move it into the static_assets folder.

Within the OutdoorMovieTheater component, we will nest all of our scene components.

Let’s go ahead and create the MainMenu scene component in a file called MainMenu.js which will be contained within the Scenes folder:

First, we need to import react and react-vr predefined components that were installed automatically installed for us via npm when we initialized our project the React VR command line interface:

import React from 'react';
import {
Text,
View,
} from 'react-vr';

Then, we need to the shell of the MainMenu component:

//imports here
class MainMenu extends React.Component {
render() {
return (

)
}
}

Our MainMenu scene is only going to contain a nested component called MainMenuContainer. The MainMenuContainer will be a layout component that will contain our Title and Button element components like so:

Therefore, we can finish off our MainMenu scene component by adding the following:

class MainMenu extends React.Component {
render() {
return (
<MainMenuContainer/>
)
}
}

Cool! We’ve created our first component in a separate file. Now, how can we make it exportable so we can import and nest this scene component in our app component?

It’s really easy, we add at the bottom:

//imports here
//class here
module.exports = MainMenu;

The syntax is simply this:

module.exports = [insert name of class]

Now that our MainMenu component is exportable, let’s import it into our index.vr.js file and nest it in our app component which again is called OutdoorMovieTheater.

First, we add the following import:

//other imports
import MainMenu from './Components/Scenes/MainMenu.js';

Note that the from path is the relative path from the file importing it.

Next, we can nest our MainMenu component in OutdoorMovieTheater:

export default class OutdoorMovieTheater extends React.Component {
render() {
return (
<View>
<Pano source={asset('fort-night.jpg')}/>
<MainMenu/>
</View>
);
}
};

Awesome!

Let’s finish things off by creating our MainMenuContainer component and our Title and Button components using the same process.

Create a file called MainMenuContainer.js within the Layouts folder.

We want to add the import, class, and export code like so:

import React from 'react';
import {
Text,
View,
} from 'react-vr';
//Layout
class MainMenuContainer extends React.Component {
render() {
return (
<View style={{
flex: 1,
width: 5,
flexDirection: 'column',
alignItems: 'stretch',
layoutOrigin: [0.5, 0.5],
transform: [{translate: [0, 0, -5]}]
}}>
<Title/>
<Button/>
</View>
)
}
}
module.exports = MainMenuContainer;

In the render function, we are returning a View component that is styled to have a Flexbox layout. Within the Flexbox layout, we will nest our Title and Button elements. Again, we make this component exportable.

Next, let’s import this into our MainMenu.js file since MainMenuContainer is already nested there:

//other imports
import MainMenuContainer from './Layouts/MainMenuContainer.js';

Now, let’s create the files for the Title and Button element components which are nested within MainMenuContainer.

First, create a new file called Title.js within the elements folder and add the following code:

import React from 'react';
import {
Text,
View,
} from 'react-vr';
//Element
class Title extends React.Component {
render() {
return (
<View style={{ margin: 0.1, height: 0.5}}>
<Text
style={{
fontSize: 0.5,
fontWeight: '400',
textAlign: 'center',
textAlignVertical: 'center'
}}>
Outdoor Movie Theater
</Text>
</View>
)
}
}
module.exports = Title;

The code above adds a Flexbox item which contains text that will display: “Outdoor Movie Theater”.

Then, create another file called Button.js within the elements folder and add the following code:

import React from 'react';
import {
Text,
View,
VrButton
} from 'react-vr';
//Element
class Button extends React.Component {
render() {
return (
<View style={{ margin: 0.1, height: 0.3, backgroundColor: '#1AC8F7'}}>
<VrButton>
<Text style={{fontSize: 0.2, textAlign: 'center'}}>
Select a Movie
</Text>
</VrButton>
</View>
)
}
}
module.exports = Button;

In this code, we add another Flexbox item that contains a VrButton with the following text: “Select a Movie”.

Note that we had to import the VrButton predefined component in this file like so:

import {
Text,
View,
VrButton
} from 'react-vr';

To finish off our first scene using a module system, we need to import the Title and Button elements into our MainMenuContainer.js file like so:

//other imports
import Title from './Elements/Title.js';
import Button from './Elements/Button.js';

If you save your files, run npm start (if you haven’t already), and go to the local host, we can now see our Main Menu scene displaying:

Woohoo!

Our index.vr.js is now much more clean:

import React from 'react';
import {
AppRegistry,
asset,
Pano,
Text,
View,
} from 'react-vr';
import MainMenu from './Components/Scenes/MainMenu.js';
export default class OutdoorMovieTheater extends React.Component {
render() {
return (
<View>
<Pano source={asset('fort-night.jpg')}/>
<MainMenu/>
</View>
);
}
};
AppRegistry.registerComponent('OutdoorMovieTheater', () => OutdoorMovieTheater);

We have now successfully organized our project. If you get confused with the hierarchy of importing and exporting, remember that our project structure provides a visual representation:

Enhancing Our Experience with Sound

When it comes to virtual reality, it isn’t too hard to understand the concept of 3D imaging as we have seen with our panoramic photos. We click to look around in 360 degrees and see different angles of the panoramic photo.

When it comes to sound in virtual reality, we can make use of 3D audio.

Essentially, we can concentrate different sounds in our 360-degree virtual world.

For example, we can have a concentration of a sound at the front of our virtual world:

Then, if we spun around 180 degrees, we could have another concentrated sound:

In the front of our app, we are going to have one ambient sound. In the back of our app, we will have another ambient sound.

This won’t necessarily require separate audio files since the L/R stereo balancing of each channel within a sound is what controls the concentration of sound in the 360-degree world.

Pretty awesome! Before we start implementing this in our Outdoor Movie Theater app, I want to show you how to I generate ambient sounds.

Generating Our First Ambient Sound

I am going to be using a tool called Ambient Mixer. This will allows us to create an atmosphere of sounds.

Here’s the link that will take you to the atmosphere creator.

As you can see, we can load different ambient sounds into our atmosphere in different channels.

Let’s start by clicking Load in channel 1:

There are a handful of predefined sounds that we can choose from.

Let’s add some ocean wave sounds found under Nature/Water/Lakes & Oceans/ocean waves.

You’ll notice that the front of our VR app looks out into an ocean.

Therefore, let’s adjust our channel so the ocean waves sound is subtle:

You’ll notice that the L/R stereo knob on each channel is a good visualization of the concept of 360-degree sound.

Since we want the ocean waves sound to be concentrated at the front of our app, let’s keep the knob pointing north.

Next, let’s load a sound called light breeze which can be found under Nature/Weather/Wind/light breeze:

Let’s make this sound a bit louder than our waves and also have it pointing north:

For the back of our app, let’s play some moon/night sounds in channel 3. There’s actually a really cool predefined ambient sound that can be loaded from Nature/Other/ambiences/Moon and forest night sounds:

Let’s also add this to channel 4 so concentrate this in the back left and back right of our atmosphere:

Make sure to adjust the knobs and volume as seen above.

Alright! This completes our audio. Let’s go ahead and click Download Audio.


Edit: As you all know, I am creating this ebook as I learn and experiment. My plan was to buy the audio and share it via Evernote as the website indicated that created atmospheres could be distributed for business purposes. Unfortunately, the audio that I bought did not play any sound and I had to send an email asking for a refund.

Nevertheless, using the interface to create the atmosphere provided us with a good illustration of how to create ambient noises and control the concentration of the sound using L/R stereo. Therefore, I will keep the previous segment in this chapter.

We will use another method to create the ambient noises.


Since the Ambient Mixer didn’t work out, we will create the ambient sounds ourselves.

I am going to be getting the individual sounds from Open Game Art and creating the atmosphere using an open-source audio software called Audacity.

If you want, you will need to install Audacity.

We also need to install Lame for Audacity so we can export audio as an MP3.

Then, you can grab the sound files that I have already extracted for you here.

Let’s open Audacity and go to File → Import → Audio and select all the downloaded sound files.

Each row is equivalent to a channel in the Ambient Mixer. We will be able to control the L/R stereo for each channel.

As you can see, the files are not the same length. Therefore, we have to trim them accordingly.

Here’s an example of me doing a trim:

You simply have to click and drag to highlight part of a channel before cutting.

Once we have trimmed them to the same length, it should look like this:

Now, we there’s some sounds we don’t need. Click the X next to each channel except for amb_forest, amb_stream, and amb_wind.

Then, double click the cursor on the minus and plus bar for amb_forest:

This will allow us to adjust the Gain. Change the Gain for the amb_forest to -17.

Next, change the gain for the amb-stream to -36 and the amb_wind to -30.

To wrap up our atmosphere, let’s adjust the L/R stereo (Pan).

We can also do this by double-clicking the cursor on the bar:

Set the Pan of amb_forest to -1, amb_stream to 0, and amb-wind to 0.

amb-forest will be equivalent to turning an L/R knob all the way to the left. This will concentrate the sound to the back left of our app.

We also want to concentrate the amb_forest sound to the back right of our app so let’s duplicate this channel.

You can do this by double clicking the amb_forest channel and going to Edit → Duplicate.

Then, slide the Pan all the way to the right or double click the cursor and set it to 1.

We should now have the following:

One last step. Select all of the channels and go to Effect → Change Speed:

Change the speed to 0.25 and hit Ok.

We are now ready to export our atmosphere.

Go to File → Export Audio and save the file as fort-night-atmosphere within the static_assets folder of our project. Make sure to save it as a .mp3 type.

Hit Ok through the dialog.

Audacity will eventually bring up the following popup:

Click Browse and the .dll file can be found at the following path:

C:\Program Files (x86)\Lame For Audacity\lame_enc.dll

Now, the audio will export as an MP3.

If you were not following along, you can download the sound file here.

Implementing the Sound Component

We are going to add a predefined Sound component into our OutdoorMovieTheater component.

The important concept to understand is that Sound components need to be nested within another component. For example:

<Image source={...}>
<Sound source={{ mp3: asset('waterfall.mp3') }}>
</Image>

Also, note that we inject the source of our sound file like so:

source={{ filetype: asset('name of file in static_assets') }}

Before we nest a Sound component ourselves, we need to import the Sound component into our index.vr.js file like so:

import {
//other stuff here
Sound
} from 'react-vr';
import MainMenu from './Components/Scenes/MainMenu.js';

Note that you would also have to be careful to import asset if you were working with an empty component file.

Then, we can finally nest our fort-night-atmosphere.mp3 file in the Pano component in our OutdoorMovieTheater component:

<Pano source={asset('fort-night.jpg')}>
<Sound
volume={0.8}
loop = {true}
source={{mp3: asset('fort-night-atmosphere.mp3')}}
/>
</Pano>

Notice, I also added a value for loop and volume. Setting loop to true will override the default setting to not loop over background audio.

Volume can be set to a value between 0 and 1. By default, it is set to 1. We are going to make it play a bit softer.

For a full list of options for our Sound components, you can check the official documentation.

Save and refresh the local host. You should now be able to hear the faint background sounds.

Try facing the back of the app and you will notice the forest sounds are more concentrated:

Amazing!


The final thing to note was that we simply nested the Sound component within the Pano component and the 360-degree sound experience came to life.

This is going to be fine for our example. However, you could have a panoramic background sound and play another audio somewhere else in the virtual world like an image:

<Image source={...}>
<Sound source={{ mp3: asset('waterfall.mp3') }}>
</Image>

Like a lot of things in React VR, there is room for exploration and experimentation after we grasp the basic concepts.

As of know, we have covered the basic concepts of sound in React VR. It’s time to move on to video.

Enhancing Our Experience with Video

Ultimately, we are going have a MovieTheater scene where we can watch movies from a hovering projector like so:

For this, we are just going to place a 2D video somewhere in our virtual world.

Before we get to that, however, I want to discuss the very cool VideoPano component.

Exploring VideoPano

Just like 360-degree panoramic photos, there are 360-degree videos that can be taken by special cameras.

These are harder to come by and a bit more challenging of a workflow to create.

I do not have the equipment to be able to discuss the workflow for creating 360 videos. However, Vimeo does have a 360 starter kit and lessons if you are interested.

If you would just like to see what people are doing with 360 videos, you can follow follow the 360 Vimeo channel.

To explore 360-degree panoramic videos in React VR, we are just going to work with an existing video that you can download here. Make sure to download it at the highest resolution, put it in the static_assets folder, and rename it video-pano-test.mp4.

The best place for finding 360 videos at the moment (and where I found the one for this example) is from Vimeo users who include a download link:

Now, let’s import VideoPano in our index.vr.js file:

import {
//other stuff here
VideoPano
} from 'react-vr';

Next, we can use this VideoPano component like so:

<View>
<VideoPano
source={
asset('video-pano-test.mp4',
{format: 'mp4', layout: 'SPHERE'}
)
}
>
<Sound
volume={0.8}
loop = {true}
source={{mp3: asset('fort-night-atmosphere.mp3')}}
/>
</VideoPano>
<MainMenu/>
</View>

In the code above, we had to specify to asset, format, and layout. The layout specifies that we want this video as a sphere so we can explore it in 360 degrees:

If you go to your local host and refresh, we can test this code out:

Super cool!

We will end the VideoPano exploration here. If you are curious about the other options for VideoPano components, you can check the official documentation.

Before we continue, revert back to our original Pano component:

<Pano source={asset('fort-night.jpg')}>
//sound component here
</Pano>

Implementing Video

To finish out this chapter, we are going to implement our MovieTheater scene component:

This scene is going to contain a layout component called MovieProjector. This will be the container for an element component called Movie.

Let’s start by creating a new file called MovieTheater.js within the Scene folder and update it with the following code:

import React from 'react';
import {
View,
Video
} from 'react-vr';
import MovieProjector from './Layouts/MovieProjector.js';
//Scene
class MovieTheater extends React.Component {
render() {
return (
<MovieProjector/>
)
}
}
module.exports = MovieTheater;

Next, let’s create MovieProjector.js withing the Layouts folder and update it with the following code:

import React from 'react';
import {
Video,
View
} from 'react-vr';
import Movie from './Elements/Movie.js';
//Layout
class MovieProjector extends React.Component {
render() {
return (
<View style={{
flex: 1,
width: 5,
flexDirection: 'column',
alignItems: 'stretch',
backgroundColor: '#333333',
layoutOrigin: [0.5, 0.5],
transform: [{translate: [0, 0, -5]}]
}}>
<Movie/>
</View>
)
}
}
module.exports = MovieProjector;

This is the same Flexbox layout used in MainMenuContainer except we are giving the container a background color of #333333 .

Finally, let’s create a file called Movie.js within the Elements folder.

Let’s start by adding the following:

import React from 'react';
import {
Video,
View,
asset
} from 'react-vr';
//Element
class Movie extends React.Component {
render() {
return (
<View style={{ margin: 0.1, height: 2}}>
      </View>
)
}
}
module.exports = Movie;

Notice, we have imported the Video component and asset.

We have assigned a Flexbox item with a height of 2 meters. Since we are using a Flexbox layout that stretches, the width of the video will be handled automatically. Our video will take up the full length of this Flexbox item.

Before we use the Video component, we need to add the video we want to display to the static_assets folder. You can download it here and rename it to fireplace.mp4.

Finally, we can add out Video component like so:

<View style={{ margin: 0.1, height: 2}}>
<Video style={{height:2}} source={asset('fireplace.mp4')} />
</View>

We have used asset to get the source and defined a height of 2 meters so our video is the full length of the Flexbox item.

The last piece of the puzzle is to import our MovieTheater scene in index.vr.js and nest it within our app component.

//other imports here
import MovieTheater from './Components/Scenes/MovieTheater.js';
export default class OutdoorMovieTheater extends React.Component {
render() {
return (
<View>
<Pano source={asset('fort-night.jpg')}>
<Sound
volume={0.8}
loop = {true}
source={{mp3: asset('fort-night-atmosphere.mp3')}}
/>
</Pano>
<MovieTheater/>
</View>
);
}
};
//AppRegistry here

Notice that I have imported MovieTheater and replaced the MainMenu scene. We just want the MovieTheater scene to display for now. We will finish the logic to switch between scenes in our next chapter.

Make sure everything is saved, refresh your local host, and let’s see what happens:

Cool beans! Note that this video does not have sound. We could control the volume just like we did with our Sound component. For full options of what you can do with Video, check out the official documentation.

Final Code

Available via Github.

Concluding Thoughts

Hopefully, this was a fun chapter that taught you some new concepts when it comes to implementing video and sound in React VR. We also were able to go over the project organization. While that part may have been less exciting, it gives you some practical knowledge for future work.

In the next chapter, we are going to tighten up this VR app by adding transitions, animations, and some better styling.

Chapter 4

Chapter 4 is now available.

Support the Author

If you would like to support me as I write this book, you can purchase the book here.

The book is published through a platform called LeanPub which allows me to update my book as it progresses. Each time a chapter is added, you will be notified via email. The book will be free to read on Medium, however, purchasing it through LeanPub allows you to download the ebook as a PDF, EPUB, or MOBI file and helps support me financially.

Additionally, I have created a special package that will provide you with a secret link to a Discord server where you will be able to help influence how I write this book.

So go ahead and purchase this book on LeanPub if you would be so kind.


Cheers,
Mike Mangialardi
Founder of Coding Artist