Comparison Shopping: Searching for JavaScript Engines for Android

When your product spans web, iOS, and Android, sharing code between apps not only saves a lot of engineering time, but also results in user benefits: shorter iteration cycles, cross-platform availability, and consistency. Code sharing is particularly useful when your code requires constant tweaking. These iteration cycles typically involve manually translating code from one language to another, and can prove to be difficult and time-consuming.

At Quizlet we build learning tools — students and learners can create a set of flashcards and study them using a variety of games and memorization aides. The core code that determines whether a user’s answer is correct contains small details for different languages and content types. This is a great example of an area of our product where sharing code is particularly useful; we are able to iterate on the grading algorithm in one place, then ship to all of our apps simultaneously.

A common way to share code between platforms is to use an interpreted language such as JavaScript. For companies developing mobile apps to complement their website, the code may already exist on one platform. However, using JavaScript code on platforms that use compiled languages natively, such as iOS and Android, can be troublesome. iOS developers have an easier time on this front: Starting from iOS 7, Apple includes the JavaScriptCore engine with the operating system, as well as bindings for Objective C.

Android, on the other hand, does not have such an engine included. Fortunately there are a few open-source projects that attempt to solve the problem of interpreting JavaScript code in Java and Android applications. The integration for each solution is not completely straightforward, and the performance of each has not been thoroughly documented from any source we could find.

After investigating several of these solutions, we have found one that provides acceptable performance and device compatibility to enable us to use JavaScript on Android. First we will discuss our testing procedure used for each of the libraries we have investigated, then dive into the tradeoffs of using each library on Android. Finally, we will discuss our experience with using shared JavaScript in the Quizlet Android app in production.

Edit 12/16: Due to feedback from several readers wondering about Duktape, we looked into the library and updated this post with our findings. All tests were completely re-run to ensure consistency.

Libraries

We found and evaluated four promising libraries:

1. JS Evaluator for Android, based around the native Android WebView.
2. AndroidJSCore, a Java wrapper around the WebKit JavaScriptCore engine.
3. J2V8, a Java wrapper around Google’s V8 JavaScript engine.
4. Duktape Android, the Duktape embeddable JavaScript engine packaged for Android.

Rhino was also previously evaluated as a potential solution, but at the time we found it was not performant enough to be usable in the Android context.

Test Procedure

To evaluate the performance of each solution, we wanted to account for several factors, including speed of marshaling JavaScript into the engine, speed of execution for simple scripts, and the initialization time for the core objects used with each engine.

We set up a simple test app that has a few modules. Each module lets the user select an engine to use for the test, and enter a number of iterations to be executed serially, allowing them to stress test each engine in various scenarios. The app tracks the cumulative execution time for each iteration, and displays it to the user once all iterations have completed.

The test modules are as follows:

1. Initialization: Creates the JS context objects for each engine in a loop, and optionally shuts them down, if the library includes a mechanism for doing so.
2. Looping: Executes a loop in JavaScript that performs a simple string comparison in each iteration.
3. Loading: Marshals a 174 KB array of sample JSON data into the engine.
4. Grading: Executes the JavaScript code used for grading answers in two of Quizlet’s core study modes.

JavaScript performance testing app

There may be more considerations for other projects utilizing shared JavaScript, but in our case, this harness allows us to effectively evaluate performance. The tests executed for this analysis were performed on a Google Pixel XL running Android 7.1.1.

On top of the runtime considerations covered in the tests above, we also needed to consider the increase in APK size resulting from including each library in the app, and the reliability of each library when attempting to run shared code on various Android devices.

Code for the test app can be found on GitHub.

JS Evaluator for Android

The advantages of JS Evaluator are immediately clear. No additional native binaries are required — the library simply uses the native Android WebView class to execute JavaScript and return it to the user. This makes adding the dependency very easy for developers, and also means the APK size increase is very minimal. Including the v1.0.7 dependency in our test project resulted in only a 10 KB increase in size of the debug APK (without ProGuard). The library can be imported as a gradle dependency, using JitPack as a repository source. Instructions in the README are fairly straightforward.

For performance, we found JS Evaluator to be fairly limited. The author states that the library is not designed to handle large amounts (~1 MB) of JavaScript. We found this was fairly accurate: The library was not able to load our 174 KB test data file for more than a few iterations of the loading or grading tests. Average initialization time was 28 ms, which was fairly fast compared to J2V8.

Code execution was a bit slower than with AndroidJSCore and J2V8, clocking in around 20 ms per iteration in the looping test, and 103 ms per iteration in the grading test. A critical limitation of this library worth mentioning, is that every call to JsEvaluator.evaluate() spawns a separate JS context, meaning all code required for execution must be passed in every call. This is different from AndroidJSCore and J2V8, where objects and functions assigned in a JS context can be re-used in subsequent calls. Users requiring a large amount of JS in context may see decreased performance, as they need to concatenate all of the JS into a single string, then marshal it into the engine every time they want to execute the code.

AndroidJSCore

AndroidJSCore is an appealing library because it is a wrapper around Webkit’s JavaScriptCore library, similar to the JavaScriptCore Objective C framework included in iOS 7 and above. iOS engineers at Quizlet use the framework to execute grading logic in our iOS app, so the underlying engine is one that has already proven itself in our production code. For Android, we needed to evaluate the AndroidJSCore library to ensure the same level of reliability and performance JavaScriptCore provides for iOS.

Importing the library is just as easy as JS Evaluator — simply import Maven dependency via gradle. The first thing you will notice after importing the dependency, however, is that the APK size increase is massive. AndroidJSCore includes binaries for many CPU ABIs, specifically arm64-v8a, armeabi, armeabi-v7a, mips, mips64, x86, and x86_64. For our test app, AndroidJSCore added 40.4 MB to the APK without ProGuard.

That said, it is fairly easy to alleviate this increase in APK size by using a feature of the Android gradle plugin called APK splits. With this build option configured, gradle will provide multiple versions of your APK, along with a fallback universal APK, to upload to Google Play. Using APK splits, the size increase is reduced to between 5.3 MB (armeabi-v7a) and 6.4 MB (x86_64). That’s still a large size increase to include with an app for the purpose of evaluating JavaScript code, but it is much more tolerable compared to the 40.4 MB increase in the universal APK.

Looking at performance, AndroidJSCore tested to perform better than JS Evaluator and Duktape, but slower than J2V8. It is worth noting that the performance of the library during stress tests was a bit unstable. During the initialization test, AndroidJSCore was quick to start up (around 9 ms per iteration), but after around 100 loops creating new JSContext objects, the library stopped processing further calls.

For most use cases, re-initializing the library in a loop is not a realistic scenario, so in the loading and looping tests, we only initialized JS context objects once, and re-used them for each iteration. The looping test proved most successful for AndroidJSCore, averaging 1.5 ms per iteration. The loading test failed sporadically, executing around 7 ms per iteration when successful. Similarly, the grading test averaged 5 ms per iteration, but tended to fail between 5 and 100 iterations.

As a last note, AndroidJSCore provides an API for interacting with JavaScript objects in Java, allowing users to interact with Java abstractions for JavaScript objects, functions, and other entities.

J2V8

J2V8 is a library that provides a set of Java bindings for V8, a JavaScript engine by Google. The project seems to be in active development and well-supported. The documentation for J2V8, however, proved to be somewhat troubling during initial integration. Despite several articles and blog posts by the author linked in the README, each source seems to reference different versions of the library, and it was difficult to find the most up-to-date setup instructions for Android. In the end, we simply imported the AAR for the latest version of the core dependency (4.5.0) found in the Maven Central archives.

Similar to AndroidJSCore, J2V8 includes pre-built binaries for the two most common CPU ABIs amongst Android devices, armeabi-v7a and x86. Because only two binaries are included for this library, the universal APK is only a 7.4 MB size increase. Using APK splits, this can be reduced to 3.5 MB for the armeabi-v7a APK, and 3.9 MB for the x86 APK.

J2V8 was very performant in our tests compared to the other libraries discussed. Despite a high initialization time of around 143 ms on our test device, the execution time was faster than both AndroidJSCore and JS Evaluator. The looping test averaged about 0.5 ms per iteration, loading averaged 2.9 ms per iteration, and grading averaged 1.9 ms per iteration. And unlike AndroidJSCore, we found the V8 engine to be very stable; executing 1000 iterations of any module in our test app completed without failure.

A noteworthy “feature” of J2V8 is its insistence on remaining single-threaded. V8 instances must be accessed on the same thread on which they were created, else the library itself will throw an error. This can prove problematic if you try to initialize the V8 engine at the start of your application using an arbitrary thread, and then use the V8 instance on a different thread. An example of working within this requirement can be found in the V8 implementations of each module in the test app.

Duktape Android

This library provides a solid JavaScript engine for the Android environment. It’s a library from Square, a large contributor to open-source projects for Android, and the project is well-supported. Integration is very easy, with instructions in the README for adding a simple gradle dependency.

Duktape provides native binaries for the same CPU ABIs as AndroidJSCore, and therefore should be runnable by all Android devices. And despite including the same number of binaries, this library is actually a lot smaller than both AndroidJSCore and J2V8, with the 1.1.0 version adding only 2.1 MB to our test APK. This size increase can be reduced to between 247 and 358 KB using APK splits — not a huge hit on app size.

The context object for Duktape initialized faster than that of any of the other libraries tested, around 3 ms per iteration. This fast initialization time can be beneficial, as it allows users to shut down instances of the context object, instead of re-using them between calls to the API, risking native memory leaks.

On the code execution side, however, Duktape ranked lower than AndroidJSCore and J2V8 in terms of marshaling and execution. Specifically, it clocked in at 58 ms/iteration in the loading test, 13 ms/iteration in the looping test, and 9 ms/iteration in the grading test. Interestingly, and possibly a point for further investigation: Duktape performed better in the grading test compared to the looping test, relative to the other libraries tested. From a reliability standpoint, Duktape held up as well as J2V8, executing thousands of iterations in each test module without issues.

In lieu of the Java representation of JS objects in AndroidJSCore and J2V8, Duktape provides a facility for calling Java code directly from JavaScript, by creating a Java interface and implementation class, and setting it as a global object in a Duktape instance.

Test Results

We ran each of the tests in our test app on a Google Pixel XL running Android 7.1.1. Results are in the charts below, displayed in iterations/second (higher is better).

It’s worth restating that JS Evaluator and AndroidJSCore were relatively unreliable in testing. The numbers above are averages over as many iterations as each library could handle before throwing an error or crashing. For J2V8 and Duktape, each test was able to run for 1000 iterations or more, and did not exhibit errors.

Here’s a summary of our findings for each library tested:

Use in Production

Given the speed and reliability of code execution, we decided to try J2V8 as the shared JavaScript solution for the Quizlet app.

Our prior implementation of the grading logic was a simplistic translation of the JavaScript used in Web and iOS. To mitigate the risk of certain end user devices being unable to load the V8 binaries, we kept the original implementation as fallback logic, as well as added logging to determine the scope of devices experiencing issues.

Unit testing a native library also proved to be somewhat of an issue, as the binaries included with J2V8 were not loadable on our development machines (OSX) or CI (Linux). This was solved by copying the libj2v8_macosx_x86_64.dylib and libj2v8_linux_x86_64.so files from the respective versions of the J2V8 dependencies to our project’s JNI folder. After adding these files, unit tests ran on our local machines and CI as expected.

So far the results have been very positive, with no reported crashes or issues from users. We look forward to continue to use shared JavaScript in the Quizlet app to accelerate iterations on our shared code, and provide a more consistent experience for users on all platforms.