Extending the p5.js Friendly Error System

by Akshay Padte, Google Summer of Code 2020

mentored by Stalgia Grigg

2020 marks the Processing Foundation’s ninth year participating in Google Summer of Code, where we work with students on open-source projects that range from software development to community outreach. This week, we’ll be posting articles written by some of the GSoC students, explaining their projects in detail. The series will conclude with a wrap-up post of all the work done by this year’s cohort.

Over the course of this summer, I worked on extending the Friendly Error System of p5.js with the help of my mentor Stalgia Grigg. The Friendly Error System, or FES for short, is a component of p5.js designed to help new programmers with common errors as they get started with learning. It detects common beginner errors and mistakes and provides helpful messages to help the user resolve these.

The major goals of this project were:

1. Improve the efficiency (speed and size) of the entire system
2. Fix bugs with the existing FES
3. Extend internationalization to cover the entire FES
4. Add functionality to detect errors spelling and capitalization
5. Add functionality to capture and simplify global errors

Before GSoC

I had been a p5 user for more than a year and had come to love it and rely on it a lot for several personal projects. I started contributing to p5 around February this year, picking up issues and helping resolve them. In this brief period, I got to learn a lot about the inner workings of p5, how it initializes, how it builds, how the tests run, how different components work together, etc.

Work

Part 1: Addressing known problems with the FES

I kicked off the summer by addressing the issue of speed and size. The FES has a component called validateParameters(), responsible for checking if the arguments passed by the user are correct. It does this by matching the arguments against a file auto-generated from the inline docs. Earlier it was imported directly into the main library for the FES to use, but it also has a lot of information that is not needed by the FES which increases size unnecessarily. Pre-processing this file to keep only what was needed helped reduce the size of the final built p5.js library by around 25 percent.

The observed reduction in the size of the library

Another issue was speed. validateParameters() involves doing extra work before the actual function is executed. Sometimes, it would slow down a function by as much as 10 times. My initial assumption to speed it up did not work so I played around in Chrome DevTools to figure out what was actually happening. I learned that most of the time was spent just trying to figure out the nearest matching overload intended by the user, and that this entire process happened over and over again if the function was called multiple times with the same arguments. I addressed this with a trie-like data structure, where each node represents an argument. Thus, if a function is called again with the same sequence of arguments, we don’t need to run the entirety of validateParameters(). This not only improved the speed but also prevented the FES from flooding the console on repetitive calls of the same function.

Speeding up validateParameters() by caching previously seen overloads in the argument tree. Parameter validation need not run for an overload that’s already a part of the tree. A new and previously unused overload, for eg. color(50, 50, 50, 50), will not be a part of this tree, and, hence, parameter validation must run for it.

There was another issue which caused validateParameters() to ignore the last undefined argument passed to function. This sometimes used to be a source of confusion. Fixing this was pretty easy and only involved one line of change.

Moving on. There was an issue that if one p5 function called another p5 function, validateParameters() would run both times. For example, the function saveJSON() needs to call saveStrings() to do part of its work. It forwards the arguments it receives to saveStrings(). This meant that if arguments were wrong when calling saveJSON(), we used to get two messages: one for saveJSON() and one for saveStrings(). But the user never called the latter in their code! This could lead to confusion.

The second message is uncalled-for and might be confusing.

To fix this, one can take a look at the stack trace. We need to answer: “Was the most recent p5 function invoked from another p5 function?” If so, we don’t need to display a message even if the arguments are wrong. I used another library, stacktrace.js, to help with this. Analyzing stack traces was extensively employed later-on in the project as well. We’ll come back to it later.

As a next step, Internationationalization support was added for validateParameters messages and the language of some of the messages was simplified. There were a couple of other small problems that were also fixed in this phase. You can see them in the full list of pull requests.

Part 2: New Features!

I had ideas for two new features to make the FES more powerful.

The first was to add a spell-check kind of system to the FES. Beginners often need time to understand the various naming conventions commonly used in programming, such as camelCase for identifiers, CAPS for constants, etc. And so, capitalization and spelling mistakes are very common, such as writing createcanvas() instead of createCanvas(), colour() instead of color(), etc. These kinds of mistakes are easier to resolve, as the browser would display an error pointing to the function call.

But if someone misspells a p5 entry point function (which has to be defined by the user), such as by defining preload() as preLoad() — which I learned is a very common mistake — p5 won’t be able to detect it and the sketch would fail silently, and it may take a lot of time to debug this simple mistake. Case-insensitive Levenshtein distance, calculated by the Wagner-Fischer algorithm was used to detect these mistakes. The check would run on two instances:

1. Whenever a reference error is thrown (which happens when a predefined function is called with a wrong spelling, capitalization)
2. When p5 is initialized, to detect mistakes in naming any entry-point functions (setup, draw, preload, etc.)

Here are a few example messages from this feature :

The second new feature was Global Error Catching. This meant analyzing the errors thrown by the browser and trying to match them up with helpful explanations to solve them.

The first step was to come up with a way to detect and classify errors. Detection was easily possible with the help of an error listener. To classify the errors, it was settled to use a regex match against a prebuilt lookup table. The idea was based on the fact that web browsers use template error strings to generate error messages. This means that for a given browser, all error messages of a particular kind would have a consistent structure. (This is pretty obvious I guess but I wanted to be sure. So I went through the source code of Chromium to confirm this 🙂.) We can have our own template strings with placeholders, which are then replaced with regex matching sequences (like ([a-zA-Z0–9_]+) ), and then the result is matched against the error message to detect what kind of error this is. The regex sequence also helps to extract relevant details. For example, take a look at this sketch:

function setup() {
  let a = 5;
}
function draw() {
  let b = a + 5;
}

It shows a very basic mistake with scope that a beginner can make. The error shown is:

[image description: Error message shown by the browser says, “Uncaught ReferenceError: a is not defined at draw(sketch.js:5) at p5._main.default.redraw (p5.js:56540) at _draw (p5.js:49120). Below that, is the error message shown by the FES, which begins with the hibiscus emoji, then reads, “p5.js says: There’s an error due to “a” not being defined in the current scope (on line 5 in sketch.js). If you have defined it in your code, you should check its scope, spelling, and letter-casing (JavaScript is case-sensitive). For more: https://p5js.org/examples/data-variable-scope.html. https://developer.mozilla.org/docs/Web/JavaScript/Reference/Errors/Not_Defined#What_went_wrong”]

While the browser error message aims at being concise, the FES message aims to explain the error as much as possible, and also provides links that have examples to fix this kind of error. This is more helpful to those who have just started learning to program and have not yet gotten used to deciphering error messages.

Another distinction to be made was between errors in user-space and errors that happened inside the library. These could be differentiated by seeing their stack trace. Moreover, it’s possible to simplify the stack trace itself to only include user-defined functions.

Here’s an example of an error that happens in library space:

[image description: Error message shown by the browser that reads, in red, “Uncaught (in promise) TypeError: Cannot read property ‘bind‘ of undefined,” then lists the nine functions in the code that the error effects. Below that is the “Error shown by the FES,” which which begins with the hibiscus emoji, then reads, “p5.js says: An error with message “Cannot read property of ‘bind’ of undefined” occurred inside the p5js library when mouseClicked was called (on line 6 in sketch.js). If not stated otherwise, it might be an issue with the arguments passed to mouseClicked. (http://p5js.org/reference/#/p5/mouseClicked),” with an arrow that points at “Error at line 6 in “a” in sketch.js, and an arrow pointing at “Called from line 3 in “setup” in sketch.js.”

The FES filters out all the internal details from the stack trace, making it easier to understand.

Part 3: Refinements

Sticking with the original proposal would have meant that this stage would add even more features. However, over the course of the project, I realized that those were not really a priority for p5 at the moment and that their implementation would take longer than what was previously expected. I discussed this with my mentor and we agreed to change the plan a bit. Instead, I worked on detaching translation files from the library and hosting them online on a CDN, and more thorough testing and documentation.

p5 uses a separate translation file for each language, and though presently only the English one is fully complete, it has grown a lot in size over the course of the summer (and is likely to grow more in the future). As more and more languages are added, bundling all of them into the library may have been imprudent. It was discussed that these could be separately hosted on a CDN, and p5 would then fetch the required file whenever needed. The build process was modified to remove the translation files from the final library. The translation files were then hosted on CDNJS and jsDelivr.

The internationalization code was modified to fetch translations from the internet, with the English file still built into the library as default and backup.

During testing, I came across an issue that global error catching did not run when running locally without a local server. I came up with a fix but it involved calling all user code through an extra layer of wrapper function. We discussed this and it was finally agreed that the benefit of fixing the issue was not significant enough to offset the negative effect on code readability this would have.

The final week of GSoC involved documenting all the changes that were made as part of this project.

Work Pull Requests and Issues

All of the pull requests made as part of the project can be found here:

https://github.com/processing/p5.js/pulls?q=is%3Apr+author%3Aakshay-99+created%3A%3E2020-05-04+

All of the issues opened as part of this project can be found here:

https://github.com/processing/p5.js/issues?q=is%3Aissue+author%3Aakshay-99+created%3A%3E2020-05-04+

Final thoughts

I really enjoyed working on this project. I learned a lot of new things. I would like to thank my mentor Stalgia Grigg for all the guidance and feedback throughout the project. I would also like to thank the entire Processing community for helping me with ideas, suggestions, views, etc.