GSoC 2018- Spifind & Music Mania

Ria Bhatia
Aug 10, 2018 · 6 min read
Google Summer of Code 2018

This post is regarding my GSoC 2018 project, where I developed and designed two games:- Spifind & Music Mania for kids with special needs. An introduction to my GSoC project, detailed information about both the games, implementation along with the problems faced and how I finally met the objectives are explained here. Also the future improvements possible are described at the end of this post. Hope you will enjoy!

Contributor: Ria Bhatia

Organization: Inclusive Design Institute

Mentors: Jess Mitchell, Dana Ayotte, Alan Harnum, Gregor Moss

Partners: Beit Issie Shapiro ;Holland Bloorview Kids Rehabilitation Hospital

GitHub Repositories

Introduction to my project

Eye gaze or eye tracking is a way of accessing your computer using a mouse that you control with your eyes. Use of an eye tracker allows children and adults with disabilities like cerebral palsy to use movements of their eyes to operate a computer.This technology can be used for many purposes, including playing games, accessing the internet and communicating with family and friends. An issue with most eye gaze based games is that they are often limited to very simple interfaces like “select an option” type games. This doesn’t give kids a chance to play, discover and explore.

Eye Tracking Equipment

Another issue involves visually impaired kids. While sighted toddlers are swiping and playing digital games at an early age, their peers who are visually impaired are unable to do the same. With the current games, until they have language, these children are unable to acquire digital literacy.

Interacting and being involved with the physically handicapped kids, over the past few years, has made me realize that the emotions, desires, and drives of these children are not different from those of a normal child. This was one of the reason that motivated me to work on this project. My keen interest in JavaScript was also a driving force that helped me break down the barriers and provide these kids with the same opportunities as others.

Objectives

Main objectives of this project were to develop an online game for young kids using an eye-gaze technology. This will help them to develop their skills through exploration and discovery. The ultimate aim with any student using alternative access is to enable the use of the same resources and help them gain confidence, clarity, understanding, social integration and self sufficiency.

Another aim was to develop a simple and fun game for visually impaired toddlers to help them begin a journey towards digital literacy. As children with visual impairments may not be able to learn by watching what is going on around them, they must learn “by doing” and interacting with their environment.

About the projects

SPIFIND is an eye gazed based game for kids with special needs like cerebral palsy and ASD. In the game, there is a dark room. As the spider appears, the kids dart their flashlights around the room to find and capture it. The flashlight will be operated by the movement of their eyes. When the flashlight is on the spider, the child dwells/blinks to capture it. The capturing needs to be done in a given time limit. The number of spiders captured by the child will be reflected on the scoreboard.

SPIFIND-SPOT THE SPIDER

This game aims to provide opportunities for free exploration and early control. This game helps assess the positioning and calibration skills of a child. It is also useful as a guide to determine the presence of early visual skills. It tells us how a kid reacts to animation and sound. Using eyes for expressive purposes and developing essential cognitive, recognition, comparison and searching skills in a pattern can be done with the help of this game. With motivation, perception, planning and motion the children can develop control over their muscles.

Play game here- Link-1/ Link-2 (initial loading may take a few seconds).

Use a free eye tracking software to play game.

MUSIC MANIA is a simple game where kids can play music using the keyboard keys. Music is a wonderful resource for an infant born blind. Exposure to it enhances a child’s natural ability to decode sounds and words. When the baby presses the key, it produces a sound. Eventually, the child will figure out that he is making that sound himself and this is an important step towards mental development. Pressing any key produces musical sound which gives a sense of accomplishment and instills happiness in the child.

MUSIC MANIA

Toddlers sway and move their hands in response to music they hear and enjoy the process. It not only improves their motor skills but is also a step towards digital literacy. The first step is understanding the location of different keys on the keyboard which will be accomplished by this game. Later on, this knowledge will help them explore and discover different technologies.

Play game here.

Implementation & Technical Details

Both of the games are web applications and are developed using standard web technologies HTML, CSS & JavaScript. For better maintainability and composability of code, I have used popular modern JavaScript framework- ReactJs. Babel has been used to transpile the code to support all browsers. For bundling of the web application, I have used webpack bundler. It is a very popular bundler with loads of plugins like uglify/minify, css imports etc. More information about the technology stack can be found here for SPIFIND and here for MUSIC MANIA.

Challenges Faced

  • Usually kids with special needs have difficulty in reading and understanding basic instructions. To make them understand the game and to think of innovative solutions to do this was really tough and challenging. I have created interesting visual tutorials with animated gifs to help these kids easily understand the game.
  • Designing the scenes and deciding the appropriate animations and music required a lot of research .
  • Arranging an eye gaze equipment was a big challenge. The US based Tobii group helped me with this and sent me a complimentary eye gaze for my project. I am really grateful for their kind help and support.
  • Kids with special needs have special requirements. A number of design changes had to be implemented time to time to make the game fun and accessible for the users. When facing these challenges, my mentors were very supportive.

Further Enhancements

I plan to improve Music Mania and will try to make it more meaningful, interesting and accessible.

Pillars of Success- My mentors!

This was the first time I participated in GSoC. Even though I had some previous experiences working with open source communities, I could never imagined getting this much exposure.

The support and responsiveness of my mentors made it easy for me to achieve milestones within the timeline. Meeting them every week and discussing my progress and problems helped me a lot to get things moving. Always providing quick reviews to my doubts, they made it really easy for me to solve issues which occurred during implementation. I cannot thank them enough for always guiding me and helping me emerge as a better programmer.

Finally, I would like to thank the Inclusive Design Institute for giving me this wonderful opportunity . Without my mentors, IDRC team & Beit Issie Shapiro Team, I wouldn’t have been able to complete this project. I hope that I was able to give something back to the community. I’m planning to contribute to the community in near future and provide help in whatever way possible.

Thank you Google!

Doing a big open source contribution was one thing I always dreamed of. Without google I wouldn’t have gotten this opportunity to meet these amazing people and get this invaluable experience. Thank you Google for giving me this opportunity!

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade