Intercept all requests in WebView on Android

The Problem

I have a requirement that all HTTP requests from a WebView on Android need to be handled locally. For example, providing assets for HTML rendering as well as handling API requests without an Internet connection. I also do not have control of what HTML content is loaded in the WebView, but with a single URL as an entry point.

Version 0.5 — override URL loading

Let’s say the requirement is to redirect the home page whenever the error page is about to load. I could use the following code.

webView.webViewClient = object : WebViewClient() {
    override fun shouldOverrideUrlLoading(
        view: WebView,
        request: WebResourceRequest
    ): Boolean {
        return if(request.url.lastPathSegment == "error.html") {
            view.loadUrl("https//host.com/home.html")
            true
        } else {
            false
        }
    }
}

As the name indicates, shouldOverrideUrlLoading returns whether the URL loading should be overridden. If the function returns true, the WebView will abort the load for the request passed into this function.

Issues

• it does not catch POST request.
• it is not triggered on any resources loaded inside the page. i.e. images, scripts, etc.
• it is not triggered on any HTTP request made by JavaScript on the page.

Version 1.0 — redirect resources loading

webView.webViewClient = object : WebViewClient() {
override fun onLoadResource(view: WebView, url: String) {
        view.stopLoading()
        view.loadUrl(newUrl) // this will trigger onLoadResource
    }
}

onLoadResource providers similar functionality to shouldOverrideUrlLoading. ButonLoadResource will be called for any resources (images, scripts, etc) loaded on the current page including the page itself.

You must put an exit condition on the handling logic since this function will be triggered on loadUrl(newUrl). For example

webView.webViewClient = object : WebViewClient() {
override fun onLoadResource(view: WebView, url: String) {
        // exit the redirect loop if landed on homepage
        if(url.endsWith("home.html")) return
// redirect to home page if the page to load is error page
        if(url.endsWith("error.html")) {
            view.stopLoading()
            view.loadUrl("https//host.com/home.html")
        }
    }
}

Issues

• it is not triggered on any HTTP request made by JavaScript on the page.

Version 1.5 — handle all requests

webView.webViewClient = object : WebViewClient() {
    override fun shouldInterceptRequest(
        view: WebView,
        request: WebResourceRequest
    ): WebResourceResponse? {
        return super.shouldInterceptRequest(view, request)
    }
}

This is a very powerful callback that allows you to provide the full response on any request made on the current page including data:and file:schema. This will catch requests made by JavaScript on the page.

This function is running in a background thread similar to how you execute an API call in the background thread. Any attempt to modify the content of the WebView inside this function will cause an exception. i.e. loadUrl, evaluateJavascript, etc

For example, we want to provide a local error page.

webView.webViewClient = object : WebViewClient() {
    override fun shouldInterceptRequest(
        view: WebView,
        request: WebResourceRequest
    ): WebResourceResponse? {
        return if (request.url.lastPathSegment == "error.html") {
            WebResourceResponse(
                "text/html",
                "utf-8",
                assets.open("error")
            )
        } else {
            super.shouldInterceptRequest(view, request)
        }
    }
}

Another example, we want to provide a user API response from our local DB.

webView.webViewClient = object : WebViewClient() {
    override fun shouldInterceptRequest(
        view: WebView,
        request: WebResourceRequest
    ): WebResourceResponse? {
        return if (request.url.path == "api/user") {
            val userId = request.url.getQueryParameter("userId")
            repository.getUser(userId)?.let {
                WebResourceResponse(
                    "application/json",
                    "utf-8",
                    ByteArrayInputStream(it.toJson().toByteArray())
                )
            } ?: WebResourceResponse(
                "application/json",
                "utf-8",
                404,
                "User not found",
                emptyMap(),
                EmptyInputStream()
            )
        } else {
            super.shouldInterceptRequest(view, request)
        }
    }
}

Issues

• There is no payload field on the WebResourceRequest. For example, if you want to create a new user with a POST API request. You cannot get the POST payload from the WebResourceRequest

Version 2.0 — resolve payload for POST requests

There are a couple of ideas on StackOverflow for this problem. One of them is to override the XMLHttpRequest interface in JavaScript. The basic idea is to override XMLHttpRequest.send record the send payload and retrieve the payload on shouldInterceptRequest. There are 3 parts to this solution.

Part I — JavaScript override

XMLHttpRequest.prototype.origOpen = XMLHttpRequest.prototype.open;
XMLHttpRequest.prototype.open = function(method, url, async, user, password) {
    // these will be the key to retrieve the payload
    this.recordedMethod = method;
    this.recordedUrl = url;
    this.origOpen(method, url, async, user, password);
};
XMLHttpRequest.prototype.origSend = XMLHttpRequest.prototype.send;
XMLHttpRequest.prototype.send = function(body) {
    // interceptor is a Kotlin interface added in WebView
    if(body) recorder.recordPayload(this.recordedMethod, this.recordedUrl, body);
    this.origSend(body);
};

This code snipper override XMLHttpRequest.open and XMLHttpRequest.send functions to pass the HTTP method, the URL and the HTTP payload to a Kotlin function. Save this code snipper into a JS file in the assets folder and load it into the WebView using

webView.evaluateJavascript(
    assets.open("override.js").reader().readText(),
    null
)

Part II — Kotlin class with @JavascriptInterface

class PayloadRecorder {
    private val payloadMap: MutableMap<String, String> = 
        mutableMapOf()
    @JavascriptInterface
    fun recordPayload(
        method: String, 
        url: String, 
        payload: String
    ) {
        payloadMap["$method-$url"] = payload
    }
    fun getPayload(
        method: String, 
        url: String
    ): String? = 
        payloadMap["$method-$url"]
}

This class will receive the recordPayload call from the previous JS code and put the payload into a map. We can add an instance of this class into the WebView using

val recorder = PayloadRecorder()
webView.addJavascriptInterface(recorder, "recorder")

Part III — Retrieve the POST payload

webView.webViewClient = object : WebViewClient() {
    override fun shouldInterceptRequest(
        view: WebView,
        request: WebResourceRequest
    ): WebResourceResponse? {
        val payload = recorder.getPayload(request.method, request.url.toString())
        // handle the request with the given payload and return the response
        return super.shouldInterceptRequest(view, request)
    }
}

This part is very similar to the version 1.5 approach except we can get the recoded POST payload from our Kotlin JavaScript class.

Issues

• For Android API 24+, the state resulted from evaluateJavascript does not persist across pages. This means any new page loaded will not have the JavaScript override from Part I.

Version 2.1 — ensure JS override available on every page

webView.webViewClient = object : WebViewClient() {
    override fun onPageStarted(
        view: WebView,
        url: String,
        favicon: Bitmap?
    ) {
        webView.evaluateJavascript(
            assets.open("override.js").reader().readText(),
            null
        )
    }
override fun shouldInterceptRequest(
        view: WebView,
        request: WebResourceRequest
    ): WebResourceResponse? {
        val payload = recorder.getPayload(request.method, request.url.toString())
        // handle the request with the given payload and return the response
        return super.shouldInterceptRequest(view, request)
    }
}

WebViewClient provides onPageStarted which is invoked every time a page is starting to load on the WebView. We will execute the JS override code snipper on every page start.

Issues

• onPageStarted is not invoked when a page is loaded into an iFrame inside the current page.

Version 2.2 — Inject JS code into each HTML page

The only function that is invoked on almost every type of request is shouldInterceptRequest. However, we are not allowed to execute any JS code inside this function since it is running in a background thread. My solution to this is to inject the JS code into the HTML content in WebResourceResponse returned from shouldInterceptRequest.

For example, you can add the JS override code into your HTML directly.

<html>
    <head>
        <script type="text/javascript" src="file:///android_asset/override.js" >
        </script>
    </head>
</html>

You could also inject the JS code into the HTML page dynamically.

webView.webViewClient = object : WebViewClient() {
    override fun shouldInterceptRequest(
        view: WebView,
        request: WebResourceRequest
    ): WebResourceResponse? {
        val resp = getResponse(request)
        if(resp.mMimeType == "text/html") 
            injectJs(resp)
        return resp
    }
}

But keep in mind that

• inject the JS code at the top of the HTML for it to take effect ASAP
• do not parse the full HTML content since it is not efficient and the HTML may be invalid

You can probably do a text search on <head> and append the script line after that.

If you enjoy reading this post, please applaud using the 👏 button and share it through your circles. Thank you.