Udacity: VR High Immersion Developer Nanodegree

Green Fingers — An educational narrative experience in VR

Cordula Hansen
Aug 22 · 8 min read

Introduction

To finish the four month VR High-Immersion Nanodegree with Udacity, course participants complete a capstone project with a number of set requirements and some extra challenges.

In this article, I am going to outline the project requirements and some of my solutions as well as the design process for this six-week project.

Project requirements

As a way of focusing what could have been a very open-ended project, the course material included a project brief: Broad enough to allow for students’ own exploration, but setting specific design and technical challenges that encouraged us to learn additional skills beyond the scope of the course.

Theme

The overall project brief was to design and develop a VR experience for the Oculus Rift or HTC Vive with the theme of “Emotion”. This could be any game, tool or narrative experience evoking a feeling in the player.

Production

The submission was to include a pre-production document outlining scope and development, a video outlining how the project meets the requirements and a text document explaining this as well.

Technically, the finished interactive build needed to meet certain specifications, such as using anti-aliasing, running at 90 fps and including at least three spatial audio sources.

In addition to the general requirements, the project brief included three categories of criteria or achievements that needed to be fulfilled: Fundamentals, Completeness and Challenges.

Pre-production and documentation

According to the course materials, the main purpose of the pre-production document was to determine the scope and main features of the experience.

For this project, I decided to also document some design decisions, a statement of purpose and a progress timeline, so that the document would help me improve and most importantly finish the project on time.

Articulating the theme and target audience helped focus the artistic style and narrative design of the experience:

Pre-production document: Outlining the target audience, overall design and timeline.
Pre-production document: Mood board and notes on art style and assets.
Production: “Shopping list” for required assets.
Pre-production document: Addressing the Udacity project requirements.
Production document: Project timeline (including a week’s break). The total production time for this project was just over six weeks in part-time mode.

The Experience: Active Optimism

The completed VR experience for HTC Vive, “Green Fingers — Restorer of Planets” is aimed at younger audiences, many of whom are concerned with the current environmental crisis. The demo represents the “training” part of a potentially expandable game, in which players take on puzzles and engineering challenges to restore and regenerate a depleted ecosystem.

Following the theme given in the brief, “Emotion”, the experience seeks to reconcile the potential angst surrounding environmental issues felt by younger audiences, acknowledging the need for catharsis, but also encouraging positive action and optimism.

The narrative acknowledges the audience’s concerns, and then guides them through a series of problem-solving activities, culminating in a sense of achievement and readiness for more challenges.

Gameplay

At the beginning of the experience, players find themselves in a bleak, asteroid-like environment within the ruins of an ancient civilisation. The mood is uncertain, a feeling of loss and seeking purpose. However, glowing gems in the environment are the key to unlocking the purpose of the player’s presence.

By activating a screen structure, player trigger a video narration, explaining the mission and alerting players to the tools at their disposal.

“Ancient Ruins” with active video player. The narrative introduces the player to their mission.

A mission list UI first explains the tasks, and on completion
articulates the positive consequences of the player’s actions.

Mission list UI: White text represents objectives and instructions, green text stands for completed tasks and their positive consequences for player and environment.

In this brief experience, three challenges are presented to the player, in part designed to showcase the technical challenges required in the project brief.

Forest cleanup

Objective for the “Forest clean-up” challenge.

A physics-based task. The player must remove plastic rubbish from the environment into a recycling device. The more plastic is removed, the more the environment recovers.

Good aim is needed for recycling “Green Fingers” style!

To encourage clean-up actions over random throwing of rubbish, a re-set script was attached to the bottles: Missing the recycler or intentionally throwing the plastic into the forest re-sets it to its original position. Some bottles are easy to reach, others are trickier to get to, perhaps in the pond.

During gameplay testing, users found this task challenging, but entertaining due to the sound effects, physics materials used and the rewards of an improved environmental aesthetic.

Environmental art as reward — flora was modelled specially for this project in Tiltbrush. The copy on the mission list changes from instructional to motivating and optimistic once the task is complete.

Solar array repair

A simple physics-based objective, where the player must first find their soldering tool and then repair electrical defects in a solar installation.

Solar repair job complete — but where does the electricity from this array go?

This tasks acts primarily as a narrative device to evoke a feeling of agency and feeling connected to the “bigger picture” in the player. During testing, players described the experience as moving and empowering. Being able to “send a message” through the positive — if somewhat mundane — activity of maintenance is a metaphor for the applicability of the VR experience to real life. This was easily understood by the gameplay testers.

The “Solar Array” environment showcases clean energy sources, and allows the player to send a message through their own positive actions.

Bee whisperer

As one of the fundamental project requirements was to “evoke empathy in the player”, I decided to design a task that allowed players to connect with a living creature in a simple way.

In “bee whisperer”, players use voice controls to guide the insect from flower to flower, sharing a sense of achievement with another entity.

In order to ensure accurate speech recognition, two-word phrases were used instead of single words, so “go back” instead of “back”. In tests, this worked reliably, as long as a short gap was left between phrases.

Bee, tree and flowers modelled in Tiltbrush. Six directional voice commands can be used to guide the bee towards the flowers.
Through voice command, the bee moves and “collects” the flowers.

In keeping with the environmental theme of the experience, the reward for this game highlights the vital role pollinators play, both in nature and in securing our food supply.

Fruit follows successful pollination of the flowers — the harvest is secure!

Achievements and Challenges

For each category of achievement,Udacity students needed to reach 500 points, with achievements weighing in at 100, 250 or 500 points. For this project, the following criteria were met:

Fundamentals

  • Scale
  • Animation
  • Lighting
  • Locomotion
  • Physics
  • Video Player
  • Empathy

The elements in this section ensure that the experience is immersive and user-friendly. For example, teleportation was used to avoid motion sickness in the user. By restricting the teleportation area, the experience also guides players from one important area to another rather than allowing a completely free exploration. This is particularly useful in a narrative experience.

Completeness

  • Gamification
  • Diegetic UI
  • AI
  • 3D modelling

This section allowed for particularly interesting experimentation, encouraging students to design VR interactions without using too much text. Due to the variety of gameplay mechanics in “Green Fingers”, it was necessary to include some verbal instructions. However, the use of video narrative to establish the scene proved useful here. The mission list was expanded in its function from serving instructions to the player, to recording achievements as a means of gamification.

AI use and 3D modelling were taken as aesthetic challenges in this project. Rather than relying on asset stores, it was faster to create most of the 3D assets in VR, e.g. Tiltbrush, Blocks and MakeVR with some texturing or re-texturing in Blender and Photoshop. With the exception of the bottle, recycler, solar panel and satellite dish models from Google Poly, all 3D assets were made in-house.

Terrain modelling, Tiltbrush assets and texturing in Blender to create almost all 3D assets for this project.

Finally, creating a flocking algorithm to give the impression of living creatures in the forest was an interesting way of expanding the code base for this project. A 2D algorithm was adapted to a 3D environment in this project and optimised for VR performance.

A glow worm created in Tiltbrush acting as a flock agent in the forest scene.

Challenges

  • Speech recognition

The speech recognition challenge in the project brief inspired the “bee whisperer” gameplay. Using Cortana, it works only with Windows 10, but was relatively easily implemented. Since the experience is designed to run on the HTC Vive, it is likely that users will have Windows 10 installed and will be able to access speech recognition as required.

Achievements playthrough

Video explaining achievement requirements and their solutions.

Evaluation and reflection

Due to the variety of project requirements, the VR High Immersion Capstone proved to be a great learning exercise. The learning outcomes were well balanced between technical, content and design challenges, which helped me try out lots of new things.

In some cases, the technical challenge also inspired gameplay, as in the pollination game, where players make friends with a bee and steer it around flowers with voice commands.

With a short deadline looming, it was challenging to create an appealing, consistent art style while also covering all the technical requirements and developing a moving environmental narrative.

Therefore this project has been a great way of sketching out a prototype for a narrative experience, but I will now continue to re-work it into separate episodes, re-visiting visual styles and interactions to create a greater sense of immersion.

If you’re an educator interested in environmental themes, I would be interested in your feedback via comment here or on social media.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade