A test strategy for updating and maintaining mobile usability of p5.js — Part 3: The final review
For the last three months I have been working on an open source project I proposed to The Processing Foundation as part of the Google Summer of Code 2018 that proposes to design a test strategy to maintain and update mobile usability of the p5.js library. I reviewed my first and second month in two separate Medium articles previously. This is a continuation of my work for the last month as well as a review of the entire experience.
Creative coding workshop
One of the highlights of the last month was hosting a p5.js mobile workshop in Port Elizabeth on July 26th to not only get feedback from users in the field who were using p5.js for the first time, but also to find any further issues with the mobile usability of p5.js.
The workshop was a 4 hour beginners creative coding workshop for high school girls with varying experience with computers and electronic devices. Students had an opportunity working with p5.js on desktop using Google Chrome as well as on Samsung tablets and their own personal devices. The learning materials for the workshop can be found here and are free to use and reproduce. This was a modification of the DIY Girls Creative Coding workshop syllabus.
From the workshop I found further issues with using p5.js on tablets regarding use of the accelerometer/gyroscope. This is an existing issue that has been reported for specific devices that I am also working on resolving as part of my project.
In addition to this, though the girls really enjoyed getting to learn coding theory and skills that are transferable to other programming languages, they expressed that they would like tutorials to be available on the website for absolute beginners to learn coding concepts through p5.js.
Overview of Summer of Code project
I started out this project on May 14th with the goal of designing a test strategy to extensively test and update the p5.js mobile functionality so that compatibility issues can be tracked easily with updates to the library and mobile platforms. The major part of my project involved performing my own visual tests on various devices both real and virtual.
During the first month of my project I spent time looking further into the types of devices that access the p5.js website and what browsers they use. Here I found out that about 45% of devices accessing the website via mobile are iOS devices, 54% are Android and the remaining 1% are various other operating systems. Also, among these devices, the vast majority access the website via Google Chrome, Firefox and Safari. With this information, I explored the best options for live testing and screenshot generation with cross-browser testing suites. These tests would supplement my live testing on my own two Android devices as well as allow me to test the p5.js website on iOS devices that I do not have access to. After looking at various testing suites such as BrowserStack and Sauce Labs, I finally decided on using TestingBot.com which provided the most recent versions of the browsers currently being used by mobile devices.
Over the second month of the summer of code, I spent a lot of time generating screenshots of the examples loaded on up to 4 different virtual devices. A virtual iPhone 6, iPad 2, Galaxy S4 and Pixel 2. This group of devices is slightly different from what I had intended on testing initially but it does cover a scope of different sized devices on both Android and iOS. While generating these, I noted new issues that would need to be resolved during my project. These included responsiveness issues with the p5.js website on mobile devices and accelerometer issues that are already listed among the Github issues noted on more devices.
The final month found me cleaning up the remainder of the mobile issues on Github and completing the spreadsheet documenting all the visual tests I performed on different devices for the p5.js library. Finishing this, there was one important consideration proposed by my mentor, Lee Tusman.
How can someone other myself take what I’ve done over the three months and use this to work on the p5.js mobile issues?
I look at my work now and see that this isn’t an ultimate solution, but a proposed intermediate step in allowing for better identification of mobile issues. Whether or not I’m the only one using it, the details provided from the screenshots and spreadsheets serve as a good indicator of where we can find the issues with the widest effects on usability. Currently these are the mobile responsiveness and accelerometer/gyroscope usage on various devices.
Challenges faced
- Cross-browser testing tools can be costly and also each are quite different in the features they provide at different costs.
- The variety of mobile devices and browsers from which p5.js can be accessed makes it a difficult task testing and assessing compatibility issues.
- I had a great difficulty with npm and grunt dependencies when working with the p5.js website repository
What I would like to do after the project
I am definitely interested in making further contributions to the p5.js library after the Summer of Code. It’s a great educational tool and I would like to look into further improving the mobile user experience and most importantly looking further into how automated testing can be done to maintain usability.
Final Submission
The final submission for my project can be found here and contains the following:
- A pull request for the p5.js repository: This is the main component of the Work Product report required for submission and contains a markdown file with further details of the submission.
- Mobile Testing spreadsheet: This serves as a guideline to what was extensively tested and what new issues were and will be created from performing mobile tests on given sets of data.
- Screenshots: This folder contains visual screenshots of each example from the p5.js reference page as an example of visual UI tests for mobile performance. The screenshots are captured for the p5.js website accessed on iPhone 6, iPad 2, Pixel 2 and Galaxy S4.
- Content from creative coding workshop: This content was created for a creative coding workshop in Port Elizabeth South Africa teaching school girls of various backgrounds coding using p5.js on both desktop and tablet.
So how did it all go?
For my very first Google Summer of Code experience and Open Source project, I expected this to be a new challenge for me and was not expecting it to be easy. I certainly did find some parts difficult such as version control over JavaScript dependencies making it difficult to replicate projects locally. I am actually a little worried about working with JavaScript in the future and may opt for contributing through Java in my next open source project as I am much more familiar with Java and using it in group projects.
That being said, some parts were also better than expected. I learned that not all contributions to open source or resolution of issues within code require actual coding and a lot of contributions can me made through documentation and helpfully redirecting misplaced issues. Also having the guidance of a mentor made this a much more enjoyable experience as I had an overseeing sense of direction that helped me see where this fits into the bigger picture as well as someone pleasant to talk to when the going got tough.
After wrestling through some dependency issues, I’m happy to have gone through this 3 month journey in open-source and would recommend programming students from all over the world to get their feet wet with the Google Summer of Code if they would like a welcoming introduction to the art of open source.
