Debugging embedded JavaScript in an Android app using Chrome DevTools

Debugging embedded JavaScript in an Android app using Chrome DevTools

A good debugger is as essential to programmers as a towel is to folks in the universe of Douglas Adams’s Hitchhiker’s Guide to the Galaxy. Our journey to understanding V8 debugging begins with a project that runs JavaScript in an embedded V8 engine in an Android app. When it works, it’s great. But when there’s a problem with the JavaScript, it’s not so much fun. Before we could commit to using this architecture, we knew we’d have to be able to debug the JavaScript running inside of the embedded V8 engine, so I began investigating how V8 debugging works, how Chrome DevTools interacts with scripts running, and how to glue this all together. If you want to see and use the final product we developed as a result of these studies, it’s available at https://github.com/AlexTrotsenko/j2v8-debugger.

Our project uses J2V8, a Java wrapper for the V8 Engine. It’s essentially a headless Chrome that you inject scripts into, and the equivalent of JSCore on iOS. J2V8 provides hooks to inject methods into the JavaScript that can call out to native code, so doing something simple like logging can be done without much work. But debugging beyond that, with breakpoints, call stacks, and variable inspections, for example, is not implemented in J2V8. The hooks to support this were added in a recent release and this blog post explains how it works and how we implemented it.

// inside the kotlin code
v8.registerJavaMethod({ _, parameters ->
    Log.d(parameters.getString(0))
}, "nativeLog")
// inside the javascript code
nativeLog("Something to log");

Registering and using a simple logging method in J2V8

Chrome DevTools Debugging Explained

The way that Chrome’s DevTools interact with its own V8 engine is pretty simple and powerful. Under the V8 hood is an Inspector API that sends and receives messages with DevTools over a WebSocket to do everything including debugging, profiling, inspecting, and more. Each message can either be a request, a response, or an event.

V8 Requests and Responses

A request includes an id, a method, and, optionally; params. It then expects an asynchronous response matching that id with a result. The messages are not guaranteed to be returned in any order, so the id is used to match the result to the request. An example is, when the V8 Engine first starts, the client (Chrome DevTools in this example) sends requests to start the runtime and debugger. In this example, the response includes an empty result, but other requests will include data in the response.

An example of requests and responses when Chrome DevTools first attaches to the V8 Engine.

V8 Events

Events include a method and optionally params. An example is when the V8 Engine has detected a script. This can happen when a webpage is loaded with scripts or when an embedded V8 Engine executes a script.

An example of a scriptParsed event, with a followup request to get the source of the script.

Debugging Embedded V8 Scripts

V8 originally had a Debugger API for controlling all of the functionality of debugging. This was removed and replaced with the Inspector API. The Inspector API exposes a way to send and receive messages from V8 that can control debugging, as well as everything else defined in the Chrome DevTools Protocol.

This project was based on the project J2V8-Debugger, a tool written for this purpose, but using the legacy V8 Debug API and an outdated version of J2V8 (4.8.2). In the updated project, we’ve migrated to the latest release of J2V8 (6.1.0) and switched to using the Inspector API.

Since both J2V8 and Chrome use V8, the majority of the time, we simply need to pass messages back and forth. Some logic is needed to transform the messages.

High Level Overview

J2V8-Debugger from a high level overview.

At a high level, the workflow is as follows:

1. The app, on the left in the above diagram, is running the embedded V8 Engine using J2V8.
2. When J2V8 sends a message (such as breakpoint hit) the J2V8-Debugger intercepts it, makes any changes it needs to and then passes it through to Chrome DevTools using Stetho (as outlined below).
3. Chrome DevTools receives the message using V8 and the Inspector API.
4. The same happens in reverse.

One thing to note is before Chrome DevTools is started, or if it’s never started, the J2V8-Debugger doesn’t do anything and the app runs the script as it normally would.

Intercepting V8 Messages

J2V8-Debugger uses the Stetho library by Facebook to intercept messages between Chrome DevTools and J2V8. Stetho handles connecting to the WebSocket and marshaling requests between the DevTools and your J2V8-Debugger.

Stetho works by exposing a class for each domain in the Chrome DevTools Protocol (CDP). To intercept messages from the client, you implement a method in the appropriate domain class with a @ChromeDevToolsMethod annotation.

Stetho also provides an ObjectMapper utility to transform an object to and from JSON parameters for convenience.

class SampleDebugger : Debugger {
    val dtoMapper: ObjectMapper = ObjectMapper()
    @ChromeDevtoolsMethod
    fun setBreakpointByUrl(peer: JsonRpcPeer, params: JSONObject): JsonRpcResult? {
        // peer is the connection to the client
        // params are the parameters sent from the client
        // optionally convert request params to SetBreakpointByUrlRequest object
        val request = dtoMapper.convertValue(
            params, SetBreakpointByUrlRequest::class.java)
        // logic to pass request to embedded J2V8
        // logic to generate and return response
        return SetBreakpointByUrlResponse()
    }
}
// Response object must inherit from abstract JsonRpcResult
class SetBreakpointByUrlResponse : JsonRpcResult

Intercepting Debugger.setBreakpointByUrl

Note that for our purposes, we only need to intercept and send messages from the Debugger and Runtime modules.

J2V8-Debugger Workflow

The J2V8-Debugger has three states it can be in.

1. The disconnected state, when Chrome DevTools is not connected.
2. The connected state, when Chrome DevTools is connected.
3. The paused state, when a script has hit a breakpoint.

The states of the J2V8-Debugger.

Disconnected State

On initial load of the app, the J2V8-Debugger initializes and then does nothing until Chrome DevTools is opened and a WebSocket connection is made. In this state, the debugger is listening to J2V8, but any breakpoints will be responded to with a call to resume.

The following steps occur when the app is started, leaving J2V8-Debugger in a disconnected state.

1. Initialize Stetho with the Debugger and Runtime modules. These are the Domains that have messages pertaining to debugging.
2. Initialize J2V8’s V8 Inspector with a delegate for incoming messages. This enables messages from the embedded V8 engine to be sent and received.
3. Dispatch startup messages to J2V8, such as Runtime.enable, Debugger.enable, etc.

Connected State

Once DevTools is opened, J2V8-Debugger enters a connected state, meaning a WebSocket connection has been established. In this state, the debugger is listening to any calls from DevTools via Stetho and the Domain and Runtime classes.

The following steps occur when DevTools is opened, leaving the J2V8-Debugger is in a connected state.

1. J2V8-Debugger receives for a request from DevTools to Debugger.enable
2. J2V8-Debugger sends a Debugger.scriptParsed event to DevTools with scriptId(s) to use for debugging.
3. DevTools sends a request for Debugger.getScriptSource and J2V8-Debugger returns script sources for each scriptId. These are the scripts displayed in DevTools.
4. Optionally, if any breakpoints are set in DevTools for the scriptID, requests are sent to Debugger.setBreakpointByUrl
5. J2V8-Debugger forwards requests and parameters to J2V8 to set breakpoints in the embedded engine.
6. J2V8-Debugger returns an id for each breakpoint.

Paused State

While J2V8-Debugger is in a connected state, requests can be sent to J2V8 and DevTools as needed. Once it enters a paused state, messages must be queued for J2V8 and sent. J2V8-Debugger enters a paused state when a script is run and a breakpoint is hit. It remains in this state until the debugger sends a resume message.

The following steps will put the J2V8-Debugger in a paused state.

1. The app calls V8.executeScript
2. J2V8 sends the event Debugger.scriptParsed and the debugger stores the scriptIds. This will be mapped to the scriptIds used in Chrome DevTools
3. J2V8 sends the event Debugger.breakpointResolved or Debugger.paused
4. J2V8-Debugger replaces the scriptId in the params with the scriptId used in ChromeDevTools
5. J2V8-Debugger passes the message and params to Chrome DevTools
6. J2V8-Debugger listens for DevTools messages that don’t have custom logic and don’t have a response (such as Debugger.stepOver and Debugger.resume and queues them to be sent to J2V8 using the V8Inspector.
7. J2V8-Debugger listens for DevTools messages that requires a response such as RunTime.getProperties
8. Queue the message to be sent to J2V8
9. Wait for response from J2V8
10. Queue response to be sent back to DevTools
11. While the J2V8 engine is paused for debugging, it will continue to call the V8Inspector delegate method waitFrontendMessageOnPause. This is where queued messages will be sent to DevTools and J2V8 while the debugger is paused.
12. When DevTools sends a message for Debugger.resume, the debugger passes the message to J2V8 and goes back into the connected state.

Using J2V8-Debugger

Script Provider

The ScriptProvider is used to send scripts to Chrome DevTools for debugging. J2V8-Debugger will intercept calls to Debugger.getScriptSource and return scripts provided by your ScriptProvider implementation.

class SampleScriptProvider(private val context: Context) : ScriptSourceProvider {
    override val allScriptIds = listOf("sampleScript")
    override fun getSource(
        scriptId: String
    ): String {
        return context.assets.open("$scriptId.js").bufferedReader().use {
            it.readText() 
        }
    }
}

Initializing J2V8-Debugger

Initializing J2V8-Debugger is done in two parts. First, we initialize Stetho with a concrete ScriptSourceProvider and the context of the app.

val scriptProvider = MyScriptProvider(context)
StethoHelper.initializeDebugger(context, scriptProvider)

Initialize Stetho with a ScriptProvider and the app context.

Then we create the v8 instance with debugging enabled. After it is created, the v8 instance is exactly the same as it is without the debugger. No additional code is needed.

private val v8: V8 by lazy { v8Future.get() }
private val v8Debugger by lazy { V8Debugger() }
   
private var v8Future: Future<V8> =
    if (BuildConfig.DEBUG) {
       // To attach J2V8-Debugger
        v8Debugger.createDebuggableV8Runtime(v8Executor)
    } else {
       // To use J2V8 without debugger
        v8Executor.submit(Callable {
            val v8 = V8.createV8Runtime()
            v8
        })
    }
fun main() {
    v8Executor.submit {
        // v8 instance is the same object after creation
        v8.executeScript(...)
    }
}

Initialize the v8 instance to use the debugger

Conclusion

J2V8 provides a way to run JavaScript in a native Android app, and, with J2V8-Debugger, you can use Chrome DevTools to debug the running scripts. Hopefully, this post will help illustrate how DevTools works and how to use V8 to debug your own code.