The Making of Microsoft Adaptive Accessories
How we’re designing assisted devices to give everyone access to the future.
By Tracy Jones
In 1980, when Bill Gates set out to put a computer on every desktop and in every home, the Americans with Disabilities Act wouldn’t be passed for another 10 years. Making computers and software for people with disabilities was a marginalized practice. As the digital world rapidly developed, the disabled community had limited access to what would ultimately change our reality. Their hyper imaginative contributions to innovation have still yet to be given a proper chance, but soon, like buildings without wheelchair access, that will be history. Last month we released Microsoft Adaptive Accessories, made up of the Adaptive Mouse, Adaptive Button, and the Adaptive Hub. They are like adding customizable ramps and handrails to stores and public spaces. The devices assist people of all unique abilities in ways that other people take for granted, things like moving the cursor across the screen, double clicking on an icon, or copying and pasting text.
The Almighty Trilogy
The Adaptive Mouse seamlessly attaches to Shapeways 3D-printed mouse tails of wildly different shapes. The Adaptive Button can be repurposed as a joystick or a dual button. The Adaptive Hub, intended to replace or augment a regular keyboard, is like the trio’s brain. Through the Adaptive Hub, the Adaptive Button can be easily programmed with keyboard inputs and shortcuts. The trio is a harmonious mobile wireless ecosystem connected wirelessly or through the Adaptive Hub’s USB-C ports. Microsoft Adaptive Accessories can be zip-tied or taped onto wheelchairs and walking aids. Any other assistive devices can also be used with a wide-range of non-Microsoft accessories. Think of the system as open-source.
The assisted devices are descendants of our company’s long commitment to the disability community, going back to 1990, when Microsoft’s own Disability Employee Resource Group was first founded. In 2016, when we started co-designing our products with customers, it was considered radical, but to us it was a natural evolutionary step towards enhancing, expanding, and democratizing UX design. Principal UX Researcher Bryce Johnson would co-design with the disability community to invent the Xbox Adaptive Controller. The release of the device was a paradigm shift, sparking other assisted devices like the Adaptive Kit, and now The Adaptive Accessories.
The Power of a Parent’s Love
The revolutionary development of Microsoft Adaptive Accessories wouldn’t take 1,000 people marching to the White House, demanding that the Americans with Disabilities Act be passed. The Adaptive Mouse was conceived by a father’s love for his child and a team of designers that were passionate about assistive devices. “I called Microsoft Adaptive Accessories the spiritual successor to the Xbox Adaptive Controller,” said Johnson.
In the Netherlands, Microsoft Senior Designer John Helmes’ seven-year-old daughter Jara is among the one billion people around the globe living with disabilities. She has cerebral palsy. The condition is a series of disorders that affect her ability to move, balance, posture, and talk. When Jara was born, doctors thought that she may not make it, but Jara’s will to live wasn’t trying to hear that. Through therapy, and with the aid of talented doctors, Jara was becoming just another girl adding magic to the world. The only problem was that she needed help operating her computer.
Using a tablet was child’s play. Through plastic holes fixed onto the screen, she could accurately touch where she intended. Using a trackball to interact with her computer was ok, but not great, barely going beyond the default. “I saw the trackball wasn’t very efficient. I thought there must be a better way,” Helmes said. From his experience, other assisted devices were difficult to find and expensive, too, defeating the purpose of accessibility. They were as subpar as Jara’s ankle braces — cold, metallic, heavy, uncomfortable, pricey.
From 3D-printed orthotics to mouse extensions
Inspired by the innovation that 3D-printed orthotics was unlocking, Helmes had Jara’s lower legs scanned. Using a digital model of her legs, he used a 3D-printer to make Jara’s new customized ankle braces out of a nylon material that was light, soft, and a lot less difficult to integrate. They were low cost, more customizable, and easily repairable. Complete with smiley faces, the orthotics were hot pink and purple with inner openings that alleviated painful pressure points along the soles of her feet. She successfully wore the braces for a year and a half. “I thought, ‘hey, if you can make an orthotic for your leg, why can’t you make an orthotic for your mouse?’” Helmes said.
He started sketching augmented mouse iterations. He had an intimate understanding of his daughter’s needs: Jara’s condition disrupted her motor skills and caused light spasms, forcing her fingers to slide off the surface of the mouse. Her hand needed something that could keep her fingers in place while navigating the cursor. Helmes, having little private resources or a budget, said, “You get a lot of ideas when you’re limited.” He looked at different kinds of pre-existing mice to figure out the most ideal shape to augment.
An old arc mouse had “a really nice small compact core,” he said. He took the tail off, short-wired the mouse’s switch, and added a 3D-printed adapter in the back of the core, behind its replaceable battery. The adapter enabled him to slip in a customized 3D-printed extension. It was his first original Adaptive Mouse prototype. The extension was intended to keep Jara’s fingers in place as she maneuvered the device. Helmes had his daughter test the prototype. It would be the first time that he saw Jara independently use her PC. She was five. To help take their Adaptive Mouse further, Helmes submitted his project to Microsoft’s Hackathon. Soon, the two-person team of Helmes and Jara would virtually multiply overnight.
A true cross-company collaboration
Each year, this week-long event brings together thousands of Microsoft hackers, tinkerers, and thinkers from around the world to participate in creating our newest and greatest inventions, like the Xbox Adaptive Controller. Through the event, Helmes, who works on Azure, met Paul Kos, Director of Electrical Engineering for Devices on the MACH (Microsoft Asia Center for Hardware) team in Shenzhen, China. His team builds most of Microsoft’s keyboards, mice, and other accessories. From the beginning, they were Helmes’ biggest supporters and closest collaborators. Johnson, Accessibility Program Manager Solomon Romney, the wider Xbox Adaptive Controller team, and many from the Windows and Devices team would also join Helmes’ project.
Together, they set core design elements. The Adaptive Mouse had to be affordable, scalable, easy to use, adaptable, and the Adaptive Mouse had to look and feel like it belonged adjacent to a Microsoft Surface tablet or any of our other hardware. Jara, who is seven now, used the original augmented mouse prototype as her go-to for operating her PC. She used it so much that it eventually had to be taped together. The team understood that the Adaptive mouse had to be as small and as thin as possible, taking up the least amount of surface area on a desktop. With Microsoft Director of Ergonomics Edie Adams helping to test and refine the comfortability of the team’s many designs, their mouse extension explorations were like the stars. Legos, forks, foam, cardboard, corkscrews, clay, and 3D-printed shapes turned into trial extensions. People will eventually be able to create and use their own personal 3D-printed extensions. Nobody tells you how to mouse was the Hackathon team’s battle cry. They produced an official mouse core prototype with a scroll wheel, left-right clicks, and an adapter extension in the back similar to Jara’s homemade Adaptive Mouse.
Co-designing to create universal access
Kos knew that solely focusing on the Adaptive Mouse was like selecting who would have access to the future. “If you go too far down a path, you don’t recognize who you’re excluding. You have to always remember to judge your decisions based on who you’re excluding,” Bryce said. He brought in people from the disabled community to talk to the team about broadening the project. “Co-creating with these communities because of their lived experience makes them the prime experts on how to design for innovation,” said Inclusive Design Director Christina Mallon. Since both of her arms are paralyzed, she uses the Adaptive Mouse with a foot extension.
The team ultimately decided to create an ecosystem, designing a switchbox hub, a multi-sensory button in addition to the Adaptive Mouse. “Some people may struggle even with that Adaptive Mouse, or some people can move the mouse but they won’t be able to press the buttons,” Kos said. After the Hackathon, developing the Adaptive Mouse was no longer play-play. The game had changed. Now, their project was becoming a reality.
“We really wanted to make sure that we’re designing a system for the unexpected, or designing for solutions that we haven’t thought of,” said Design Director Simon Dearsley. His department was tasked with refining the Adaptive Mouse and getting the project onto the accessories roadmap. The Adaptive Hub and the Adaptive Button’s features and functionalities were mostly realized, though figuring out Adaptive Mouse solutions was like spiraling through a worm hole.
Where’s the juice?
Debate over maintaining the Adaptive Mouse’s replaceable battery or implementing a rechargeable one led the team to consider developing their own battery, but it would need certification, taking up a lot of time and resources. They integrated an inner rechargeable battery with a USB-C port so the Adaptive Mouse, Adaptive Hub, and Adaptive Button could share the same battery. The design saved a lot of money, but how were people going to recharge the mouse while using it?
Putting the USB-C charging port at the core’s tip was a start, but what if the Adaptive Mouse tilts towards the desk? What if a person’s extension slants the Adaptive Mouse at an angle, preventing it from laying flat on the desk? And was the original adapter extension still an attractive solution? Without an extension attached to the adapter, there was a large empty slot cut into the back of the Adaptive Mouse, making it looked unfinished.
The team’s iterations involved various looped tails, magnetized lids, and top parts, that successfully contained the core. Jara, her occupational therapist, and her classmates with varying disabilities got to see, touch, feel, and try them firsthand. Beta testers who worked on the Xbox Adaptive Controller tried them as well. Through their trials, Helmes found that the process of changing extensions and using custom extensions while recharging became difficult. Though the loop tails showed promise, all the iterations were either too complicated, too fragile, or they needed additional parts.
The team landed on attaching the back of the core to extensions, clicking them together. If the extension tilts the Adaptive Mouse off the desk, there’s an optical sensor at the bottom curvature of the Adaptive Mouse that prevents the core from losing tracking capability. Dearsley thought of giving the core’s tip a chamfer, adding a slope allows the Adaptive Mouse to be tilted or angled without damaging the USB-C port connected to the cable charger, which allows people to use the core with any extension while charging the Adaptive Mouse.
Everyone is a world builder
The image of paralyzed protesters climbing the steps of the United States Capital would forever put dips in curves and crosswalk sounds on street corners. Those protesters showed how they were disabled by a world designed against them, but like Jara, they would not settle. As a Microsoft designer, Helmes used his expertise to help his daughter, which catalyzed a company-wide effort to help many. As the digital world continues to be built, Microsoft Adaptive Accessories will empower everyone on the planet to take part in constructing a technological universe that will be designed by you and for you.
To stay in the know with Microsoft Design, follow us on Twitter and Instagram, or join our Windows or Office Insider program. And if you are interested in working with us at Microsoft, head over to aka.ms/DesignCareers.
