What I Built (3) — Browser Extension for Intercept and Detect Requests’ Country of IP Address

This is one of the learning about what I can do with a browser extension, the extension is Firefox only for now as I leverage DNS resolve capability that is not supported in Chrome (yet).

dns.resolve()

Intention and Background

As data privacy are of more concern (at least I do concern), I would like to know what kind of request is made underneath when I browse on the internet (with my browser). There is manual way to inspect those underneath requests, for example, using browser developer tools check the network, then nslookup the IP address, then request the origin from some provider/API to resolve the country origin.

the developer.mozilla.org is hidden behide cloudfront and the IP in above nslookup does not resolve in this API, this is a failed example…so using 8.8.8.8 to screen capture

What I built

Notification pop up when a site make request to some county code

Zoom in notification (sample from yet another site)

Getting start

The reference I follow is the MDN document:

Intercept HTTP requests

My manifest.json is as follow:

{
  "description": "Checking webRequests geo location",
  "manifest_version": 2,
  "name": "check_geo",
  "version": "1.0",
  "permissions": [
    "webRequest",
    "webRequestBlocking",
    "dns",
    "storage",
    "unlimitedStorage",
    "notifications",
    "<all_urls>"
  ],
  "background": {
    "scripts": ["background.js"]
  }
}

For permissions:

• the core item to intercept web request is “webRequest” and “webRequestBlocking”.
• the “dns” is the permission to use dns.resolve()
• as I add notification capability, so I need the “notifications” permission
• finally I need to reduce the hit to location API, so whenever the domain is hit before, it’s being cached in local storage, and thus I need “storage” permission, while the “unlimitedStorage” is not actually needed (depends what I want to store)
• and I want the addon to apply to all URLs, so the <all_urls> is added

The core code logic in background.js is to register an event listener:

browser.webRequest.onBeforeRequest.addListener(
  logURL,
  {urls: ["<all_urls>"]},
  ["blocking"]
);

The above code add event listener before all web request, and call a handler “logURL” and make this apply to all urls, the config is in “blocking” mode, which imply the request would not sent until the handler call finish, I use the blocking approach because:

1. I used to want to reject the request if it’s not what I want it to request
2. Even I allow it pass through, I want to cache all the domain I have did the location API check because all those API are of limited hit per minute / per month, and a modern website would hit a lot of request even it look idle (e.g. for tracking or improve UX purpose like record where you stopped reading/viewing last time)

The logURL handler should be something simple as (skipping the localstorage code and exception handling here):

async function logURL(requestDetails){
  // Get hostname from requestDetails object
  const hostname = (new URL(requestDetails.url)).hostname;
  // DNS resolve
  const ip = await browser.dns.resolve(hostname);
  // check location API
  const ipapiUrl = `https://ipapi.co/${ipAddress}/json/`;
  const response = await fetch(ipapiUrl);
  const result = await response.json();
  const country_code = result.country_code;
  
  // if country_code = "XX", notify
  if (country_code === "XX"){
    browser.notifications.create("XX", {
      type:"basic",
      title: "XX site",
      message: `Origin Url: ${requestDetails.originUrl}\n\nUrl: ${requestDetails.url}`
    })
  }
}

The Tricky Part

The tricky part is, because the event listener is applied to all urls, so the call to ipapi.co to get location information is also part of the event, and this make an infinite loop of creating ipapi.co call and checking the ipapi.co call.

Ended up, my solution (I am not sure if it’s the best practice) is to RegEx check the URL before the core code above, with calling a function like this one:

function excludeUrl(requestDetails){
  if(/ipapi.co/.test(requestDetails.url)) {
    return true;
  } //else
  return false;
}

Conclusion

This addon implementation is far from perfect, there are following things I can see it cannot resolve yet:

1. ipapi.co does not need to sign up and is free, but one cannot hit it too often (officially 1000 call per days, but when hit too frequent, it would become access deny), one can use a paid version of such API and there are more choices out there, as well, the paid plan can perform bulk lookup.
2. most of the popular domain are having a local server around your country, so the dns resolved IP might be local (to your country), to resolve this, one can extend this to also match a blacklist of domain name, but I bet there are alot of extension helping one to manage that.