APDE GSoC ’18: Preview Mode, Incremental Compilation, and Wrap-up

9 min readAug 14, 2018

This is the second and final post about by experience in the 2018 Google Summer of Code. See my previous post for background and more information.

Get the code: one huge diff for all GSoC code changes.

Here is the status of all planned features:

Android mode 4.0 integration (wallpaper, watch face, and VR support): fully implemented and released to production (v0.5.0). Diff. Branches: android-mode-4 (merged), android-mode-4-watch-send (merged), android-mode-4-watch-embed (dangling, unused).
Preview mode (run sketches without installing them): fully implemented and currently testing on the preview channel (v0.5.1-pre3). Diff. Branches: preview (active, ready to merge)
Incremental compilation (show errors and warnings in real time): in progress, scope expanded beyond original plans so was unable to finish in time. Diff. Branches: incremental-compilation (merged), preprocessor-rewrite (active).
In-app tutorial system (for users new to APDE): not implemented. This project was deemed unnecessary after the initial proposal.

Quick Links:

Google Play (download)
GitHub (open source, wikis, issues, download, GSoC diff)
Preview Channel (preview releases)

General Status:

Android mode 4.0 integration is fully rolled out to production and is currently installed on 7000+ devices. There were some hiccups during testing, but that’s what the preview channel is for. v0.5.0 is very stable, even with all of the major changes required to add the different targets.

Preview mode is finished and currently undergoing testing on 70+ devices on the preview channel. I fixed a bug on 4.4 earlier today, and if everything else goes according to plan, I should be able to start rolling these features out to production in less than a week.

Incremental compilation is still in progress. Basic error reporting and highlighting is implemented — i.e. APDE shows you your compilation errors in your code with red and yellow underlines, just like a real IDE. The build process is not yet daemonized, however. Additionally, the preprocessor problems are not yet integrated into the same reporting interface. While working on this feature, I realized that the existing preprocessor was inadequate (only capable of reporting one problem at a time, and not able to recover from simple mistakes to produce further compilation problems). Thus I decided to rewrite the preprocessor. I did not originally plan for a preprocessor rewrite, so I wasn’t able to finish this within the time allocated for GSoC, but I am planning to continue working on it in the next couple of weeks. More information about this below.

What’s New?

For the cool new features from Android mode 4.0 integration, check out my last post. These features are now available from production, so just download APDE from Google Play to give them a spin.

Preview Mode

Preview mode runs your sketch in the “sketch previewer” without having to install the sketch. The sketch previewer only has to be installed once, and then you can run sketches as many times as you want without reinstalling anything. Preview mode is available as a target alongside app, wallpaper, watch face, and VR. Preview mode is intended to be the default mode for sketches going forward.

This is how preview mode works:

When you first run a sketch, you are prompted to install the sketch previewer. Each subsequent run does not require an install and build times are very fast.

For more information about preview mode, read further below or see the APDE wiki. You can get preview mode (v0.5.1-pre3) by opting into the preview channel (these names coincide by accident).

Incremental Compilation

This feature will compile the sketch in real time and display the compiler warnings and errors in your code as you type. It is not yet fully implemented, but the basics of detecting and highlighting problems are there already.

In addition to the problems highlighted in the code, there is a problem overview screen next to the console.

This is what this feature looks like so far:

It is still rough around the edges, but this new compiler problem workflow should make the process of fixing compilation problems much faster and nicer overall.

Implementing Preview Mode

The implementation for preview mode was based on the APDE Wear Companion for running watch face sketches because I was forced to resort to classloading for that feature. (See my previous post for more information.) It turns out that classloading is also a useful approach for regular sketches because it means that the sketch doesn’t have to be reinstalled every time.

The first solution that comes to mind is to directly load the sketch’s classes into APDE, but this is non-ideal for two reasons: crashes and permissions.

If the sketch crashes, then it crashes all of APDE and there is no way to get the stack traces. Additionally, every time that a user’s sketch crashes, that crash would get reported to the Google Play Developer Console on my end and completely drown out all of the actual crashes from APDE itself.

If the sketch were loaded directly into APDE, then the sketch would not be able to use any permissions beyond those granted to APDE. I don’t want to add too many extra permissions to APDE for concerns about user privacy.

The next-best option is to create a separate app, a “sketch previewer”, that isolates the sketch from APDE so that crashes can be handled correctly. The idea is that the sketch previewer is packaged with APDE and installed the first time that the user runs a sketch.

There is still a problem with permissions, though. Namely, all possible permissions have to be granted to the sketch previewer because permissions cannot be added dynamically after installation. There are privacy concerns associated with this approach. Further, users might want to use third-party permissions that the sketch previewer can’t possibly know about in advance.

As it so happens, APDE is capable of building full Android applications. The solution is part elegance and part jank: build the sketch previewer within APDE on the fly to add new permissions.

The way it works is as follows: the sketch previewer is compiled on my end, but not packaged into an APK. The intermediate build files are packaged into APDE. Then when the user runs a sketch in preview mode for the first time, APDE finishes the build and installs the sketch previewer. Whenever the user runs a sketch that requires additional permissions (e.g. camera), APDE rebuilds the sketch previewer, injecting the new permissions into the manifest, and reinstalls. Thus the sketch previewer must be reinstalled whenever the user runs a sketch that requires permissions that have never been used in preview mode before. The sketch previewer accumulates permissions over time and requires reinstallation less and less frequently.

For more information about preview mode, see the APDE wiki.

Basic Problem Detection and a New Preprocessor

I want to start off by pointing out that using the phrase “incremental compilation” to refer to this feature is perhaps misleading. It might be better termed “real-time compilation” or “active error checking”, but neither of these sound quite as good and I have already named the branch incremental-compilation.

Getting the problems from the Eclipse compiler (ECJ) is the easy part. They have always been spit out in the console, and that was how debugging in APDE used to be before incremental compilation. It turns out that the hard part is mapping problems from the preprocessed Java code back to the Processing code.

The preprocessor combines all of the sketch tabs together and then performs some changes to convert Processing code into Java code. First, everything is wrapped in a big PApplet subclass (so Processing classes are actually inner classes), imports get moved to the top, the size() method is extracted, methods are made public, and a host of other minor changes take place. The problem is that all of these changes affect the position of problems reported by ECJ.

The Current Approach

The current way of handling problem reporting is to track a line number offset for each tab and then map ECJ-reported line numbers back to the sketch code by line number. This is effective, but only gives us line-precision, and we want to highlight bits of code within a line. The position of code within a line isn’t guaranteed, either, because the preprocessor does things like add public before every method declaration, changing the character counts.

The implementation demoed in the video above handles this problem in a non-satisfactory, but surprisingly functional way. When an error is reported by ECJ, the raw character positions are looked up in the preprocessed Java file to find the exact text to highlight. APDE also finds which occurrence of this exact text should be highlighted in this line.

For example, if the line is println(println) and the error is that the variable println is undefined, the second instance of println in the line is reported.

This approach (line number, exact text, and occurrence within the line) lets us highlight the exact place in the original sketch code, but it has a few shortcomings.

For example, consider this code:

Original: void test() { println(public); }

Preprocessed: public void test() { println(public); }

This approach will fail for the above code because the preprocessor injects public at the start of the line, meaning that the problem is the second occurrence in the Java code but the first in the original code. The current implementation doesn’t know what to do in the situation, so it gives up and just highlights the entire line.

A New Preprocessor

The problem is that the current preprocessor doesn’t give us enough information about what changes it makes to the code. There are also problems in addition to those described above, such as the inability to highlight size() or import statements (they get extracted by the preprocessor).

Another issue is the fact that the preprocessor only reports one error at a time. (ECJ reports all compiler problems at the same time.) This means that a simple syntax error, like a missing semicolon, prevents all other problems from being detected.

Finally, the current preprocessor is a Frankenstein mess of code from the Processing project and random patches for APDE compatibility that is difficult to maintain. This situation is non-ideal, and thus warrants a re-write.

Like most of the things that I have been doing with APDE, this is a problem that has already been solved on the desktop. In this case, Manindra Moharana’s PDEX (a GSoC ’13 and ’14 project) served as inspiration. He used Eclipse’s JDT DOM package for preprocessing instead of ANTLR, as all versions of Processing had done before. This served as the basis for my new preprocessor implementation.

Much of this preprocessor has already been implemented for the desktop, but it proved infeasible to integrate into APDE because it was too tightly coupled with the desktop Java mode, so I am in the process of creating my own implementation, tailored specifically for APDE. I intend to decouple the preprocessor sufficiently from APDE so that it could potentially be backported to the desktop in the future.

JDT DOM can report all errors at once (just like its sister, ECJ), and also has the ability to recover from basic errors (e.g. a missing semicolon) so that the code can still be compiled. This way the user can see all problems with their code at once without having to fix preprocessor errors first.

At the core of the new preprocessor is a system of applying and replaying edits to translate sections of code from the original to the preprocessed code, and vice versa. This way, problems reported by ECJ can be mapped back to the original sketch code directly, without having to deal with line numbers or line occurrences.

The preprocessor is currently mostly working, but a lot of testing and integration work is needed before it will be ready for the preview channel. I am going to keep working on the preprocessor and incremental compilation in the time that I have left before school starts.

All together, this will be a much improved build system. I did not originally intend to rewrite the preprocessor this summer, and that is why I did not finish everything that I had planned. But I believe that the reasons outlined above justify this departure from my plans because this approach will lead to a much more solid final product.

Incremental Compilation + Preview Mode = The Dream

Once the new preprocessor is complete, I can turn my sights to the true goal of incremental compilation: near-instantaneous builds. The idea is that preprocessing, compiling, and dexing will run as a continous daemon as the user types, so that the sketch can run instantly when the user presses the “run” button. This dream is only feasible thanks to preview mode and other features implemented this summer.

Build times on my phone are already down to less than two seconds with preview mode enabled, so these features, when finalized, should take APDE to the next level.

Fin

Please check out preview mode from the preview channel and all of the work on GitHub. Thanks for reading! That’s all.