Design For Wellbeing
David Abraham, John Amaral, Praneeth Appikatla, Yixue Wang
If you want to check out our product for yourself, click here.
Introduction
Most video games are by nature made to support our wellbeing. However, sometimes these games can serve to detract from our wellbeing and leave us more frustrated and angry than we were before. For this assignment, we decided to use the Affectiva library to read in user emotions while playing a game and promote overall wellbeing using these emotions. Our final product provides one possible solution to minimizing the frustration sometimes produced by games.
Design Process
Brainstorming
As a group, we originally came up with two different ideas: recommending a video to watch based on the user’s emotion, and suggesting taking a break to the user. We ended up going with the latter option since it seemed to promote well-being more than the other idea.
We brainstormed different ways to have our system recommend the user to stop whatever it was they were doing and take a break. Ultimately, we decided on using a game that players would easily get frustrated at. That way, our system would inform the player about their rage and encourage them to step away for a few minutes and recollect their thoughts.
At first, we thought about using a game like Fortnite, where the player would be looking at a TV screen or monitor that was displaying the game while having the laptop with the integrated webcam separately recording their face. However, we felt that this was not very intuitive, as having two separate devices performing two different tasks would feel a bit clunky.
In the end, we came up with the idea of integrating some JavaScript code into our HTML page to have a game running in the same page that the camera operated with. This would make it possible to have both the game and the camera running on the same device, which would make our system so much easier to use.
We also decided on having different text display on our website depending on the emotion read in by the camera. For instance, if the player showed disappointment, the text would console the player and tell them not to worry, whereas if the player displayed anger, the text would tell the player to take a break.
Research
The question remained as to what we would use for our HTML game. We thought about finding some JavaScript code online to modify for our own use. As such, we found some JavaScript code for a Flappy Bird-esque game. We figured this was a perfect game to use since players would get easily frustrated if they accumulated a really high score and then hit a wall at the last possible moment.
We went through the tutorials that the website provided to actually generate the code of our game. Originally, the bird was a generic yellow square, and the walls to pass through were generic green squares. We wanted to change these to something more unique that could cause users to react with many different emotions.
Somehow we thought of using the phenomenal actor Tim Allen as a basis for our game. We replaced the generic yellow bird with a picture of Tim Allen, the generic green walls with pictures of houses (as a reference to the fantastic television series “Home Improvement”), and the jumping sound with Tim Allen’s grunt from “Home Improvement.” We even added a noise for when the player hit a wall, which was one of Tim Allen’s quotes as Buzz Lightyear from “Toy Story.”
This idea ended up working out in the end, as at first the game would elicit laughter from players, as the sounds of Tim Allen are comedic. However, after pressing the spacebar to jump multiple times, the player would become extremely annoyed and frustrated by the constant presence of the Tim Allen grunt noise, prompting us to add a mute button for the game.
Demo
The first iteration of our website was created for our demo day. It was very simple, as it contained the game on the very left side of the screen and some plain text in the center. We included the title, a play and pause button for changing the play state of the game, a high score counter, and text that changed dynamically with the emotion read in by the camera.
The design was so simple because the game originally did not have its own HTML element. It was generated by running the game script that contained the corresponding JavaScript code. Therefore, we could not move the game window at all, so we simply put our text off to the side. This was not a major concern for us during the demo, but was something that we would act upon for our final product after we received feedback from users.
We ended up integrating a high score system that kept track of the highest score that the player had for one particular session. If the player refreshed the page, the high score counter would be reset to 0.
The major focus of our demo was the text that changed depending on the emotion read from the camera and interpreted via the Affectiva library. We simply set up an HTML element dedicated to this text, and it would change based on the emotion from the user. Since the camera takes a few seconds to load up on start, we had a “Loading camera…” text as a placeholder until the camera started up, in which case the text would state something along the lines of “let’s play!” (The text is based around the voice of Mr. T to elicit even more laughter from players). We also designed the background color of the website to change with the text to correspond with the emotion detected.
If the emotion detected by the camera was rage, the camera would stop and the game would pause. The background color would turn red, and the text would inform the player to take a break and come back in a few minutes after they were feeling refreshed. Then, the player would start the game up again and continue playing. This was our main focus for our system, which is why it has so much more functionality compared to everything else.
We decided to hide the camera from the player because there is no point in having the camera display while the player is playing the game. In fact, a camera would only make a user more nervous and distracted. We simply made a JavaScript method to hide the camera, which also meant less space was taken up on the website. Instead of having separate start, stop, and reset buttons for the camera, the “play” button started both the game and the camera, and the camera stops when it detects anger from the player. The camera starts back up again when the player clicks on “Play.”
Feedback/Final Product
On the demo day, we received a plethora of useful feedback from other groups. People generally enjoyed our system, but one major change they recommended was adding additional elements to our website, as there was a lot of empty space originally. Essentially, our website needed more interactive designs and more instructions to teach the user how to start and play the game.
To make our game easier to understand, we added instructions that informed the player to click the spacebar to make the bird in the game jump. We originally assumed that players would understand this control scheme and that we would not have to explicitly say it on the website, but we ended up including it because there was a decent amount of confusion.
We also added a short description that described how the camera and the Affectiva library worked together to read in user emotion. This informed the players that their faces were being recognized by the camera, and it gave the players an understandable explanation of how the website was changing based on their emotions displayed.
To make the the website have more interactivity, we added a GIF for when the player got angry. (Of course, the GIF had to be of Tim Allen screaming maniacally with a chainsaw in his hands.) The GIF provided an even greater sense of humor to the website. Although the GIF displayed anger, it was a comedic element that made players feel better about themselves.
Another major element of the “angry state” of our website was that the “play” button was disabled for 5 seconds. This was done for two reasons: one, to prevent the user from accidentally clicking the spacebar mid-jump after they were angry and causing the “play” button to be pressed again (as the focus of the mouse would be on the “play” button if that was the last button the player clicked); and two, to give the user a little bit of time to read the text and calm down.
Since we added more elements to our page, we had to redesign the layout of our website to make it look more flushed out. We did this by dividing the page into 3 sections: the left section for the high score counter and the program description, the middle section for the game title, play, pause, and mute buttons, the game itself, and the instructions, and the right section for the dynamically-changing text and the angry GIF. In the HTML code, we created containers and put different elements in those containers to organize the layout of our site.
Conclusion/Reflection
There were a few things that we considered integrating into our application that we simply did not have enough time for. For instance, we could have changed the difficulty of the game based on the user’s emotion, such as increasing the gaps that the player has to jump through when they were frustrated.
We also wanted to create a better UI in general for our site by adding more interesting text fonts and having even more interactive elements. In the end, we kept it simple because it was not our main focus and because of the aforementioned time constraints. Some of the feedback we received from the demo day included some of these ideas,most of which we did end up including in our final product.
Looking back on our work, we designed a really fun, interactive application that promoted the wellbeing of our users. After some technical difficulties, we were able to utilize the Affectiva library to read in user emotions and change our website to correspond with these emotions. As a result, we believe that our site helps to promote overall wellbeing.
If you want to check out our Flappy Bird clone for yourself, click here. Thanks for reading!