Classroom Interaction Tool for students with cerebral palsy

Jean
Jean
Jul 21, 2018 · 4 min read
Image for post
Image for post
Source: http://engineeringgood.org/local-programmes/

Many students in the school run by the Cerebral Palsy Alliance in Singapore have drastically different cognitive and physical abilities. Within the school, they are assigned to classes based on their cognitive capability. Nevertheless, some students are verbal while most of them are non-verbal, resulting in an uneven playing field during lesson time. The verbal students often answer the teachers’ questions more quickly, dominating any discussion there could be within the classroom. Consequentially, students who are more shy, or answer much more slowly, simply parrot the other students’ answers.

As part of a team of volunteers in Engineering Good, I visited the school several times to observe the lessons in action, and to talk to the teachers about their experiences. To engage the non-verbal students better and understand their progress, we re-designed how quizzes and games could be administered.

The main idea of the design was to enable each student to answer quiz questions independently of other students, whilst fulfilling the following requirements:

  • the system had to be wireless
  • provide a way for teachers to track the students’ answers and progress
  • easily maintained by teachers/volunteers
  • customized to various levels of mobility
  • inexpensive

I. Radio frequency prototype

The first prototype we tried was using an RF module to transmit and receive signals. It was connected to an Arduino Uno which would process the signals, and cause different colored LEDs to light up based on the student’s choice. These LEDs were placed at the front of the setup, so technically, only the teacher could see which choice was made as they were all facing the teacher. The students have limited mobility and are unable to turn to see their classmates’ answers.

II. Reconfiguration of wireless mouse

A mouse typically can trigger at least three events via the left click, right click, and middle click, which is sufficient to answer a multiple-choice quiz question that has three choices. Furthermore, a commercial mouse is easily compatible with any computer, and already has inbuilt radio frequency capability to transmit wirelessly.

Hardware

I took apart a mouse and rewired it to three mono jacks, which could then be connected to a variety of switches. Students have their own favored switches for their range of tactile sensitivity.

Image for post
Image for post
Unsolder the original switches from the mouse board and connect them to female jacks
Image for post
Image for post
Rewire buttons to male jacks

Software

Each student would have a reconfigured mouse that could connect to the teacher’s laptop (Windows 7). The challenge here was then to be able to:

  • differentiate between each mouse
  • translate the mouse clicks into answers

I wrote a desktop application mainly to be able to get the device handles of USB devices, which are required to differentiate between each mouse. In a typical .NET application, I can get mouse click events, but not the device handle. Taking reference from the project Using Raw Input from C# to handle multiple keyboards, I modified it to support multiple mice instead. The principle behind the code is well explained in the code project. In summary,

  • Interface devices produce raw input, which is usually bundled up as a bunch of events and translated by the Windows manager — so all left clicks from different mice are just known as left click events, without differentiating between the devices
  • I intercept the bundling, and filter for the events that I’m interested in, and translate each event in our custom code. The handle of the device that generated the event is kept associated with the translation.

The interception code only runs when the teacher has chosen to enter ‘quiz mode’ on the application. When it is not running, the Windows manager processes the left, middle and right clicks of the reconfigured mice as per normal. As such, even without the application, the students could still use the mice to play games and train their reaction skills.

The next difficulty was in embedding a local Powerpoint file within the application. I found various suggestions online about embedding a Powerpoint Viewer, but in the end, decided to use the inbuilt WebBrowser class. The teacher could host his slides on Google Slides or OneDrive to show it in the browser, or display a local PDF or HTML file.

Further improvements

The system has been handed over to the teacher for a pilot test. One of the challenges is getting the mice to install their drivers properly on the Win 7 laptop, which can be frustrating. As the displayed Powerpoint and administrative view are on separate screens, the teacher has to constantly move his cursor from the Powerpoint , to the administrative view to enter/exit quiz mode, then move back to the Powerpoint again to go to the next slide. I should incorporate keyboard shortcuts, or move the enter/exit options to the Powerpoint view.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store