Creating a Happy Place for Pain Relief

How mimerse partnered with Sweden’s most beloved chain of pharmacies to create what might be the virtual future of pain killers.

Happy Place for GearVR: A Virtual Reality pain therapy app by mimerse and Apotek Hjärtat.

Our startup mimerse works much like an indie game studio, but rather than producing games for entertainment, we develop therapeutic apps. This summer we were approached by the pharmacy chain Apotek Hjärtat, one of Sweden's most beloved and recognized brands with 400+ retail locations. They wanted our help to come up with a way to use technology to benefit the large group in society in some way experiencing pain. Naturally, we were up for the challenge.

Psychology influences how we experience pain. Sensations of pain can be manipulated by changing the way we think and feel. Virtual reality technology, by definition, creates an audiovisual illusion displacing the user into a digitally created world, something one study found to be as effective as narcotics in mitigating pain.

Why a Virtual Reality?

A wide range of VR content has been demonstrated to reduce pain in situations ranging from toothaches to serious burns.

Initially, our working hypothesis was that the observed positive effects were due not to the design of specific VR applications — but rather an inherent capability of virtual reality technology itself.

Despite numerous research projects and companies doing pain distraction in VR — we could not find any case of a publicly available app with this explicitly intended use.

But could most VR-content be a sufficient pain therapy solution? When we browsed the rest of commercially available VR-apps and experiences, we ruled out most of it for pain distraction purposes due to the following:

  • Not being immersive, distracting or engaging enough to be effective for most users (such as 360 video content).
  • Being at risk of creating negative emotions, such as a sense of failure, over stimulation or nausea.
  • Being hard to learn, requiring too much setup or complex interactions (not likely tolerated by a person in pain).

Demands of a virtual happy place

In order of priority we believe an ideal VR-application should aim to achieve:

  • Distraction from real world and real body by maximizing presence.
  • Low risk of inducing negative emotion.
  • Accessibility in terms of design, content, technology and platform.
  • Promote positive side effects such as relaxation, calmness and feeling of awe and wonder.

The Challenge

We decided to create something that satisfied the demands of those in pain that could be made available in our partner’s primary care clinics as well as free of charge.

Our biggest challenge, besides our tight budget, would be to appeal to, and work for, as many as different types of people, pain levels and situations possible.

Influences

We drew inspiration from a wide range of fields going into the project.

Patient using Snow World — the most famous VR pain relief application so far.

VR pain distraction research

Immersive virtual reality pain distraction was first explored by Hoffman & Patterson at the University of Washington Seattle and Harborview Burn Center.

The concept has been further developed commercially by companies like DeepStream VR (who also have compiled a list of relevant research literature) and others like AppliedVR .

The Calm room — spaces for emotional self care in Sweden's psychiatry

In common practice, a distressed person should ideally be in an environment lacking stimulation. However “The Calm Room” project, a success story in Swedish psychiatric hospitals, has demonstrated the therapeutic value of adding actively calming stimulus.

Guided meditation VR and virtual nature research

Spending time in nature has therapeutic value, and there are scientists investigating if the same effect can be reproduced with virtual nature.

Edgbaston’s Queen Elizabeth Hospital using virtual nature to alleviate pain through distraction therapy.

Users widely report therapeutic effects of Guided Meditation VR, an excellent meditation app that features non-threatening, beautiful and natural environments.

Open-world games and camping

From the start we wanted to work on maximizing presence and distraction by creating something that felt like a place. To capture this feeling we decided to include day, night and weather cycles and wildlife in our environment.

Few things are associated with escaping reality like camping. Camping provides a good cue to sit still and chill and gave us a reason to add a bunch of “stuff” around the user.

Swedish children’s books

For some users, sitting in virtual nature would not be enough. We decided to fill the scene with things to see and explore to add to the distraction.

A typical Pettson and Findus scene.

“Pettson & Findus” is a series of children’s books about a cat and a man living in a ramshackle red-ochre-painted wooden farmhouse in the Swedish countryside.

The author Sven Nordqvist cluttered pages with tons of things to see and enjoy, almost like Where’s Waldo but with no clear objective. Keen readers could reread and study each page, noticing new things every time.

Hearthstone

Finally, the hidden clickable board interactions of Hearthstone (an online card-game) gave us more ideas.

These interactions are might be lost on users busy playing the game, but under- stimulated players (perhaps waiting for opponent) make it their quest to seek out these interactions on each board, adding value for those easily bored and in need of stimulation.

Visual references

Once we knew roughly what features we wanted, we collected visual references.

Landscape painting and r/earthPorn

When exploring possible settings, we browsed a lot of r/earthPorn: a subreddit dedicated to sharing beautiful pictures of nature.

Later on, we found that the slightly stylized, composed and romantic feel of paintings worked even better.

Victorian watercolor painting of a fjord by artist Hans Andreas Dahl (1881–1919)

Low poly virtual reality

A counter intuitive fact about virtual reality is that aiming for realism might not be the best choice to achieve maximum presence.

A scene in Oculus Dream Deck (a VR demo-reel) convinced us that the popular Low Poly art style was viable for our project. Despite the lack of geometric details and textures, to us it felt like a very immersive place. The art style also suited the project both in terms of budget and hardware limitations.

Oculus dream deck experienced created by Tyler Walters

Design and development

The combination of its graphical capabilities, relatively low barrier of entry and large user base (millions) made the Samsung Gear VR our primary target platform.

The Samsung GearVR: a relatively cheap, mobile VR-headset compatible with Samsungs high end smartphones.

Happy Place was developed using Unity3D.

If you are not interested in the process, consider skipping ahead to the next session.

Creating the place

A lot of effort went into designing a pleasing composition. We tried many configurations, focused on an effective default forward facing view and then added more detail.

View from within the Unity3D editor.

Every 3D asset was created from scratch by the same artist, this way we could retain a unified stylized feel in the entire experience.

Some of the 3D-assets created for Happy Place.

The art style required high quality recognizable silhouettes for the objects we created for the world. Rather than using textures, we relied on vertex coloring (a colour value stored for each face of the 3d model).

Vertex colors and custom shaders

We had to plan carefully how to spend the limited graphical resources of our target platform.

We developed a custom shader to be able to tint everything in the world on the fly without spending a great deal of computational power. This made it possible to dynamically change weather, time of day and also control the level of saturation of an individual object.

The water uses a fresnel effect with a scrolling normal map, and a subtle vertex deformation in the vertex shader. The reflection is from a cubemap generation script. Specular component has been disabled to reduce aliasing and boost performance.

Gaze Objects and interactions

We decided to have no visible UI, use no buttons for input and build everything on gaze- based interaction.

Gaze Object Interaction during the first seconds (an object triggered is coloured by the shader)

We came up with a concept we called “gaze objects”: intractable objects placed around the scene. Gazing at an object would enable a timer, and a circular shape would appear around crosshairs filling up gradually over time. Once the circle had finished filling up, the event is triggered.

We never explicitly tell the user which objects they can interact with. It is up to the user to figure out and explore through trial and error which objects trigger events.

One would be hard pressed to find all of them. We finally included 50+ gazable objects hidden in the world.

Some interactions are intuitive, like gazing at the bucket puts out the fire. Others are bit harder to find but also more rewarding. Here he come.

Sound

We created a soundscape that felt relaxing and non-threatening. Sound effects were mainly sourced from open source field recording libraries.

Furthermore, we put in a dynamic soundtrack to add to the distraction.

Having previously ported the excellent Moodelizer dynamic music engine into Unity, we felt it would work perfectly in Happy Place.

Moodelizer lets us dynamically change the tone and feel of the music on the fly. By connecting variables to events in the scene, we created a soundtrack that responds to the users actions.

The moodelizer theme used in Happy Place

We decided to include two human voices. A female voice providing the basic context and a British voice actor reading from a “bodyscan” relaxation manuscript.

Gazing at the radio and a book respectively toggles music and the guided meditation.

Problems and pain points in development

  • A bug in Unity forced us to decide between an unusually long loading time or frame drops in loading scene.
  • Performance was an issue throughout the project, but combining meshes by baking let us drastically decrease the number of batching and improve performance by almost 20%.
  • Not being able to edit vertex colors outside of Maya turned out to be a problem that could have been solved with various Unity asset store packages.

Optimization and stats

The final GearVR application takes up 78mb. The scene contains around 70.000 + triangle count on average with a small amount of textures (5+) and runs at the required 60 fps on target devices.

Desktop build

The desktop version started as an experiment to see what Happy Place would be like using:

  • Deferred rendering
  • Expensive post process effects like SSAO, proper HDR and tonemapping
  • Real-time shadows

We decided to use the desktop build for producing some of the media rather than relying on low quality captures possible in the GearVR.

Release and packaging

A GearVR app

Happy Place is now available on the Oculus Store . Download it for free.

Case films

Happy Place trailer

Two videos showcasing the usefulness of Happy Place were produced.

One video features Albin, who gets a tattoo in Happy Place and the other Sara, who seeks refuge in Happy Place from her pregnancy related pains (she gave birth later the same day).

360 video capture of the app

We documented Happy Place by producing a 360-video using the excellent VR Panorama asset and the desktop-version of Happy Place (6 hours of render time for this 5 minute video).

If you have a google cardboard, or a browser that supports youtube 360, get an immersive taste of Happy Place here.

Implemented in clinics

A systematic review of clinical trials have found that distraction can help reduce pain from needles in children. This was the use case most clear in our minds envision Happy Place being used clinically. We made a special Swedish build for partnering clinics to use and test in their day-to-day work.

Outcome

First semi-public unveiling of Happy Place. Lucky one to the right gets to try the desktop version.

Following a few weeks in development, we demoed a near complete build of Happy Place to hundreds of pharmacists at an internal conference event, a majority of whom were unfamiliar with VR.

We observed that none of the testers:

  • Had to ask how the interaction worked.
  • Reported feeling ill or any other discomfort or negative emotions.
  • Complained about it being boring.

In fact, people of all ages and backgrounds were overwhelmingly positive and amazed by Happy Place.

As expected, the way people interacted with the world was widely different depending on how much distraction they needed.

Some users were so taken up by the novelty of VR that despite spending a long time in the app, they failed to notice or pick up on the gaze interaction mechanic. They just sat and enjoyed the scene.

Some virtual friends to interact with in Happy Place.

Others closed their eyes and followed along with the guided meditation. Some immediately grasped the hidden interaction concept and started to methodologically scan the entire scene.

Unlike our other apps, we have yet to perform any scientific testing of the pain distracting capabilities of Happy Place, these will take place soon.

What we learned

We are committed to creating more apps like Happy Place.

Our latest experiment was “Itsy”, a spider phobia treatment game released on the GearVR. It was developed in collaboration with Samsung Electronics, the Swedish Government and Stockholm University. And like Happy Place, it was the first app of its kind to be automatic and commercially available.

Apps like Itsy and Happy Place are only the beginning, the first trembling experiments in a format that we think will disrupt psychological treatments once immersive and augmented reality platforms really hit mainstream.

The cost of pain related health care is nearly 30% higher than the combined cost of cancer and diabetes, estimated at nearly $300 billion annually in the US alone. The potential for technologies like Happy Place in healthcare, even if only adding value to a small subset of users with pain in need of solutions, is arguably huge.

It is important not to get too excited or carried away by the novelty of VR. Much of the work is left, for instance figuring out when, and how to optimally use apps like Happy Place.

In the recent Oculus conference an “education” section was unveiled and we predict we’ll see a “Health” section soon enough, as this is a usually overlooked but enormous use case for the technology we love.

Credits

Happy Place was developed by William & Alexander Hamilton (design and code), Jeryc Paragua (code an art) and Keith Bacad (art and animation).

Special thanks Wenderfalck (concept and PR), Apotek Hjärtat, Arthur Röing Baer, Vobling AB, Moodelizer AB, Rosko Lewis and Diane Lehman (voice acting).