UX & Barriers for Singapore’s baby boomers..?
Smart Accessibility & Mobility for Barrier-free Access (SmartBFA) aims to be the leading platform for real-time barrier-free access route advisory and navigation in Singapore. Routing and accessibility information is delivered to our user base through a dedicated iOS or Android application.
Usability test overview & Methodology
A Usability Test (UT) was conducted to investigate the user-friendliness of our developing SmartBFA mobile app. Testers were shortlisted from SmartBFA’s pool of volunteers (wheelchair & mobility scooter users) based on the following criteria:
1. Must own and use an Android or iOS smartphone as the UT aims to replicate scenarios of real users in their natural environment
2. Able to speak and read basic English as the application is currently available only in English
Usability test process
The UT process can be broken down into 3 different segments, namely introduction, task explanation & execution, and exit interview.
A short introduction is given to the volunteer to provide background context for the application, administrative information and the goal of the UT (which aims to assess how users interact with the app). Volunteers are encouraged to think out loud to share their feedback which will be captured along with how they interact with the application for internal analysis. Some simple questions such as experiences with navigation apps and comfort and time spent on the internet were also asked to help make conversations and ease the volunteer into the test.
Each volunteer is provided an identical scenario with regards to navigation to set the context. A sample scenario:
You are visiting your friend at ‘One Devonshire’. You decided to use the newly available SmartBFA mobile application to look for a wheelchair accessible route to ‘One Devonshire’.
A total of 8 tasks are then given to each volunteer. Volunteers will execute each task sequentially. A sample task:
Find an accessible route to ‘One Devonshire’ from your existing location
Upon the completion of the 8 tasks (regardless of task success or failure), an exit survey is conducted to solicit feedback. A 5-point scale is used to rate usefulness’, ‘ease of use’ and ‘recommendation likelihood’. Qualitative feedback on ‘like & dislikes’, ‘future important feature(s)’, ‘recommendation considerations’ and any other miscellaneous information was collected.
Result comparison & Analysis
Each volunteer was assigned a tech savviness index for analysis based upon the self-reported response during the introduction to:
1) Experience with mapping applications like Google Maps
2) How much time they spent connected to the internet
3) Comfort level with the internet
The categories are on a nominal scale of 1–4 with 1 being technologically illiterate to 4 being technologically savvy. However, volunteers are effectively only assigned from the range of 2 to 4 as category 1 does not make the exclusion criteria as mentioned earlier.
Each task outcome is assigned a score of either 0, 0.5 or 1 based on the outcome of success, success with issue/assistance, or failure. This is done by reviewing the captured video and hand labeled by an assessor (i.e. yours truly). The assessor also logs qualitative observations of how the users interact with the mobile application.
Due to the limitation of the study sample size (n=8), it is difficult to derive a comprehensive quantitative analysis. The results as-is suggested a weak correlation (0.51) between the performance score and assessed tech-savviness. It should be noted that the tech-savviness rating is based on a qualitative interview which may include self-reported and assessor’s bias.
We then used age groups as a more objective measure to analyze the results and found a negative correlation of 0.829! There were statistically significant differences between result score means based on generation group as determined by one-way ANOVA (F(2, 5) = 8.534, p=0.024) at the p<0.05 level. A Post hoc comparison using the Tukey HSD test indicated that the mean score for the baby boomers group (M = 2.750, SD = 1.5546) was significantly different than the millenials group (M = 6.500, SD = 0.0000).
It should be noted that the sample size is small (N=8) due to the constraint on cost, manpower and subject availability, a more comprehensive result can be expected with a bigger sample size. However, the result still provides evidence that the application is relatively user-friendly for younger users and we need to rethink our UX to cater for the older generation smartphone users.
So, how can we improve the SmartBFA application for all users?
To do so, we reviewed interaction footages and noticed some interesting patterns on how users interacted with the SmartBFA application.
- Ooh, shiny! We found that senior users tend to fixate their attention on the first field they see from the top to bottom of the screen. For example, when searching for an accessible route in SmartBFA, the phone location services provide the current location which is automatically filled in the first field as the starting point by default. Senior users tend to click into the first field to manually enter their current location again. Many then tapped ‘Choose from map’ which is the first button they see after entering the end address to try to initiate the search.
2. Scroll vs Click & Tap? Many modern applications use a combination of scrolling and tapping scrolling to organize the flow of information and try to minimize the clicks and time required to find the desired information. However, we observed that a number of senior users were struggling to identify how to proceed after certain form selections and scrolling is not apparent to them, leading them to tap on random buttons to try to complete the required tasks.
3. Expressive Icons. Our observations suggest that there may be a difference in how users interpret the meaning of icons depending on age. In the context of adding favourite locations, younger users were more likely to correctly identify the meaning of unlabelled icons than senior users.
4. Usefulness & Recommendation Likelihood. In our interview, senior users were less likely to find accessibility & mapping tools like SmartBFA to be useful and were also less likely to recommend them to others. Some initial hypothesis for this phenomenon could be due to these users’ reluctance to adopt technology or their travel patterns (which consist mostly of familiar routes & places) do not warrant a need for navigation applications. Further investigation is required to drive better adoption and value for this user group.
Conclusion
Our observations point us to a markedly different mental model of how users in different age group approach technology and user interfaces. A significant amount of effort is required to design a universal application which can cater to users from all walks of life. With the usability test, we have identified some usability improvements in our app design which we are looking to incorporate into our beta application. Solutions we are exploring are the addition of labels, an indicator for scrolling, updates to certain process flow and an introductory tutorial. Hopefully, with these additional implementations, the barrier to accessible navigation will be further lowered for all!
SmartBFA needs your help!
Are you using a wheelchair or a mobility scooter? Or know someone who is? Help make Singapore a more inclusive society by participating in the SmartBFA project! A sensor system will be retrofitted on the mobility device for a period of two (2) months to automatically collect data on path accessibility (e.g., bumps, obstacles, slopes, etc). The sensor data will be anonymized and aggregated, for analysis on the “wheelchair-friendliness” of the paths travelled. A small reimbursement token will be provided for every week of data collection. Sign up at smartbfa.com or contact us at smartbfa@gmail.com!
About author…
Songwei is a Full-stack Developer at Trampolene, a non-profit, working on the project Smart Mobility and Accessibility for Barrier-Free Access (SmartBFA).
