A test strategy for updating mobile functionality of p5.js — Part 2

This is a continuation of my first article documenting my experience in Google Summer of Code as a student working with the Processing Foundation. Read here

It’s almost peak winter in Port Elizabeth, South Africa. This makes for lots more long nights of coding and a little more time to dedicate to my Google Summer (or Winter ❄️) of Code project. It feels great to be past the halfway mark for my first time participating in GSoC. I’ve learned a lot more than expected especially pertaining to tools and project management, which should not be forgotten as part of any project.

Here’s a summary of what I got upto, which mostly involves addressing Github issues and compiling screenshots for UI tests.

Late night coding and cocoa

So far…

For the first 4 weeks I spent time designing what testing method I would be implementing for the remainder of the project. This involved looking into the scope of what I would be able to test and what tools I would be using. Here is a summary of what I was able to decide on and accomplish during the first few weeks:

• Defined test priority for this project as the p5.js examples and Github issues on Chrome, Firefox and Safari
• Decided that I will be using Testingbot for Screenshots and Live tests
• Created a spreadsheet summarizing results from UI tests with links to screenshots
• Begin addressing Github issues and adding more issues to repository from tests

For the second month, I immediately got into finishing compiling the screenshots using TestingBot. I decided on testingbot because of the accessibility to recent operating systems for testing, live mobile testing and the ability to take and download up to 45 screenshots at a time

Testingbot has a variety of operating systems for live testing and screenshots

For each example on the p5.js website, I generated 3 screenshots using iOS and Android: 1 each for iPad 2, iPhone 6 and Galaxy S4.

Github issues

The Github issues listed for this project can be divided into p5.js website issues from the website repository, and p5.js library issues that are issues within the p5.js library itself. The issues from the p5.js website repository contain most of the issues that affect mobile responsiveness. Here is a breakdown of the issues.

• Issue #3019 Mobile examples do not load on larger tablet screens:

• Issue #3020 Default canvas size for various examples too large for screens: When opening certain examples such as the Form-Regular Polygon example on mobile, the default canvas size of the sketch is too small for some mobile device screens and information is left out.

• Issue #3008 iPad external keyboard issues with keyboard.js: This is a high priority issue that was found affecting iPad users using a USB external keyboard. As shown by Chris Orban in this video, when a key press is registered on the external keyboard, p5.js does not recognize when the key is released.

• Issue #3003 displayWidth/displayHeight report incorrect values on Android: This is a reported issue noted on the Moto G (in Chrome v67 using Android 7.0) and Mac so far. For instance, when executing this sketch to test the display width and height using the p5.js library I get 1366*768, which correctly matches my current screen resolution. Other users have reported incorrect values being detected from the library.

• Issue #2751 touchStarted() breaks slider: As reported by a user, returning false on the touchStarted() function within code with a slider prevents the slider from moving. This however does seem to be expected behavior of the code but I would like to look into the user’s specific case and see if I can assist in their implementation of the slider in their project

• Issue #2218 accelerationXYZ doesn’t seem to work on Android: This issue shows up when attempting execute certain sketches that use accelerometer data on certain Android devices. You can try this sketch out to see if it works for you. This appears to be a device specific issue which I have tested live on my own devices. Both devices display sensor data using other apps, however when executing this sketch, the acceleration XYZ data works on my Android phone but not on my tablet and other devices listed by users in the issue page.

On my device (Infinix X572) the example above gives us a cube that jitters as we rotate the phone and change the accelerationXYZ values

On a Lenovo Tab S8–50, though the accelerometer is working, opening the example above gives a static cube that does not react to the phones movements

• Issue #1875 <Select> Elements not responsive on mobile device browsers: This issue is possibly related to it’s preceding issues where an input is detected twice. Select items are clicked twice which makes them appear unresponsive
• Issue #1815 mousePressed and touchStarted broken in chrome mobile: With this issue, when using the mousePressed() function in mobile, when the screen is touched, both the mousePressed() and touchStarted() functions are fired. Leading to a touch being detected twice on mobile.
• Issue #1282 issues with cross browser support for deviceMoved() and deviceTurned(): With this specific issue, the deviceMoved() and deviceTurned() event listeners on certain devices produce the following error.

In addition to compiling the screenshots on testing, for the remainder of the month I decided to tackle the mobile responsiveness issues and the high priority iPad issue. The first issue I attempted to resolve was the keyboard.js issue affecting iPad external keyboards.

Issue #3008 was reported by Chris Orban and documented in this video here. When using USB keyboards on the iPad 2, the keyPressed() function does not recognize when a key is released. Prior to receiving the video from Chris, I attempted to recreate the issue myself. In my assumption that it was an issue involving all external keyboard interactions with the iPad, I had a friend test this issue using an Apple Magic Bluetooth keyboard and failed to replicate it. This confirmed that this was an issue specific to USB keyboards. The recommended fix to try for this issue was adding event listeners to Unfortunately I had no access to a USB keyboard and iPad to continue working on this issue and it is currently being undertaken by Kate Hollenbach.

Responsiveness issues

Some of the major issues involving mobile usage of the p5.js library are responsiveness issues. The most notable issue to me was the fact that mobile examples cannot load on 7" tablet screens as they are detected by the library as larger screens. This is because the CSS code identifies mobile devices from the devices screen width. However this does not take into account larger mobile device screens like tablets. To get past this I began looking for other ways to detect when to hide and show mobile examples. From a discussion with my mentor, Lee Tusman, he recommended using the gyroscope activity to detect when the mobile examples get shown as the mobile examples do require the usage of the gyroscope. For reference, this StackOverflow solution to detecting a gyroscope within the browser is being used in my ongoing implementation of this solution.

For the rest of the month I will be doing the following:

• Clearing the remainder of the Github issues in the p5.js repository
• Creating content for and hosting a p5.js mobile session for high school students in Port Elizabeth
• Completing compiling screenshots for examples and Github issues
• Documenting the mobile usability improvements

📢Only 2 more weeks until the end of the Summer of Code 🌞

Keep a look out for my next post where I’ll be writing about our upcoming p5.js mobile workshop in Port Elizabeth and reviewing the improved mobile usability as a result of this GSoC project.