Porting Kindle Word-Wise to the Browser

I keep a list of projects I’d like to take up. This particular one has been on it for quite some time. So what is it? An extension.

Stop laughing.

The Idea

A year ago, I’d purchased Kindle. Best decision ever! As you might have guessed, it has this wonderful feature called Word-Wise.

Their website says:

“Read more challenging books

Word Wise makes it easier to enjoy and quickly understand more challenging books. Short and simple definitions automatically appear above difficult words, so you can keep reading with fewer interruptions.”

Since the past few months I have been way too into Javascript. I had been going through the HTML5 semantic elements, as one does on a Friday night. When I saw this:

The HTML <ruby> element represents a ruby annotation. Ruby annotations are for showing pronunciation of East Asian characters.

And something in my head clicked!

The idea was simple.

Procure a dictionary → Get difficult words → Traverse the DOM and annotate aforementioned words with the <ruby> tag (which is supported by most browsers) to display the definition.

Let’s begin

Look Ma! (Show Me The Code)

Voracious

Checkout the code.

The Process

I bootstrapped my extension, added Webpack for bundling and had my project set up in under 30 mins. The basic idea was that I would inject a contentscript.js into a page, analyze the text and display the definition.

Now, I needed a list of tough words. A few searches later, I had a list of 5000 words primarily used for SAT prep. A little cleanup and I had this:

{
  "abase": "To lower in position, estimation, or the like", 
  "abbess": "The lady superior of a nunnery"
  ...
}

Nice! But not enough. The list contained words such as “access” and “accept”, not what I would categorize as tough words. Each word needed a difficulty associated with it. That’s where dictionary.com comes in.

Dictionary.com provides a difficulty level for each word

So I did what any sane person would do, send a request for each word in my dictionary, parse the HTML response and find that little data-difficulty attribute in the page. All using a 70-line node script with axios and cheerio.

I ended up with a refined version of the dictionary.

{
  "abase": {
    "definition": "To lower in position, estimation, or the like",
    "difficulty": 48
  },
  "abbess": {
    "definition": "The lady superior of a nunnery.",
    "difficulty": 42
  }
}

Here comes the interesting part, the crux of the extension.

I started off writing helper functions.

So far so good. I began writing the logic for DOM traversal.

Attempt #1

This was the first approach I’d used. Be warned, it is näive and may cause heartburn.

It goes like this, I select all the <p> elements in the page, get their inner text, split the words, annotate them and set the innerHTML .

There are some major flaws in this,

1. If <p> or <li> has child nodes they are lost.
2. <p> and <li> may not be part of an article.

Back to the drawing board.

Attempt #2

Let’s go the brute force way. Why not traverse the entire DOM and annotate the text. In theory, this sounds logical.

A little background first. A node in a document can be of several types. The most common ones are Node.ELEMENT_NODE which represents an element such as <div> and Node.TEXT_NODE which is what you’d think it would be.

Node has this helpful method replaceWith() which I’m going to be honest, took me way too long to find.

The ChildNode.replaceWith() method replaces this ChildNode in the children list of its parent with a set of Node or DOMString objects. — MDN

VanillaJS FTW! Who needs jQuery am I right!

Attempt #3

Using document.body had some side-effects. The google search page would flicker like crazy (If anyone knows why, let me know, I’m curious). Also, this isn’t very efficient. Many elements that ought to be skipped such as <script> are traversed.

I decided to use some intelligence while traversing. HTML has an <article> semantic just for this. Although I found, many websites avoid using it. I ended up using these class and tag selectors.

The Result

This is far from perfect, but it’s a start. Some noticeable bugs are

1. Since I use class & element selectors, the extension fails on some websites (Clearly it’s their fault).
2. The context of the sentence is not considered, sometimes leading to the wrong definition being displayed.
3. Some words are not recognized during parsing. This happens when the word has adjacent punctuation such as . , or ( etc.
4. The dictionary is by no means complete. It has roughly 5,000 words. In contrast the Oxford dictionary has 171,476 words.

Since this extension is not dependent on browser specific api’s, it can be ported to other Browsers like Firefox, Opera without any changes, something I am actively working on (I’m at position 386 out of 488).

That’s all folks!

PS: Love the Chrome extension publishing process.

Call To Action

If you enjoyed this article, let me know by clicking the 💚.