Befriending WYSIWYG Editors: Text Highlighting with Virtual Underlines

Spelling and grammar-check is one of the requirements to processing the user’s text input. Browsers have a built-in base functionality to solve spelling and basic grammar problems. However, users always lack extra capabilities for text proofreading such as correcting spelling, grammar and punctuation mistakes, improving the style and tone of voice according to a certain domain.

Also, users abandon browser solutions due to their inability to work with the complex markup of the content of the editable element.

Broken functionality in a rich text editor with an enabled native browser spell-checker

As a result, third-party tools for proofreading texts in browsers have crowded the market.

There are two types of these tools:

• B2C browser extensions (Grammarly, LanguageTool, Linguix, Ginger, ProWritingAid);
• B2B solutions that exist in the form of plugins or scripts added to users’ pages (WebSpellChecker, ProWritingAid, NanoSpell, LanguageTool).

The former group has several advantages: fast installation, few requirements, a user-friendly interface, personalization options. The latter provides security for users’ data since the solutions can be installed on clients’ servers.

Other advantages of B2B solutions include:

• Easy integration via APIs;
• More customization options for companies and brands;
• Master dictionaries extending default dictionaries of a domain or company with specific terms;
• Domain specific dictionaries;
• Multi-language support;
• Self-hosted solutions with extra security.

This article is devoted to the forced evolution of WebSpellChecker, a spell- and grammar checking tool for web apps. We’ll be talking about our moving away from straightforward browser APIs to a custom approach. It’s based on the creation of an external layer for markup under the original editor where we add blocks positioned relative to errors in the original text.

Ultimately, this approach appeared to be viable, forward-looking, flexible, and scalable. However, it needs the support of the developers of modern editors who forced us to make such changes.

Good old way of text highlighting

How to inform users about spelling and grammar issues found in the text? You should add an additional markup that will be highlighting the text with issues. Here’s an example of how it works in one of our earlier products, SpellCheckAsYouType plugin for CKEditor 4.

1. Let’s imagine that you’ve got the following text markup:

<p>Hello <b>w</b>orldd</p>

2. In this case, there’s a spelling mistake in the word “worldd”.

3. We need to find the start and the end positions of the word “worldd” in the text nodes of the DOM model. To an end, we parse the DOM model and get all the text nodes from it.

4. Then, we prepare a <span> element by adding certain attributes to wrap the text in it. It will be used for highlighting issues found in the text.

5. And finally, using a standard Range API provided by all the modern browsers we wrap the text with issues in a pre-prepared <span> element.

6. Done. Now we’ve got the following text markup:

<p>Hello <span class=”scayt-mispell-word”><b>w</b>orldd</span></p>

7. CSS class “scayt-mispell-word” adds an underline to an incorrect word.

This a standard way of underlining errors in a text provided by all the browsers.

Standard approach to text highlighting

Another important question in highlighting problems in the text is how to remove underlines or replace a word with a correct one chosen by the user. You can do it this way:

1. You can replace the text of a <span> element with a correct one.
2. And remove this wrap (<span> element) while keeping the correct text.
3. You can use standard Range APIs to fulfill both tasks.

This is how you can add and remove text highlighting with replacing an incorrect word or phrase to a correct one.

Standard approach to text replacement

These base mechanisms work well, fast, and, more importantly, they’re compatible with all existing browsers and WYSIWYG editors of an early generation (CKEditor 4, Froala Editor, TinyMCE, etc.). The developers of third-party spell-check and similar solutions successfully used them for long.

Winds of changes

And then the day came when developers of WYSIWYG editors despite having fully working solutions decided that they shouldn’t rely on DOM and ContentEditable any more as it’s hard to combat with browsers and it’s time for changes.

Speculations to this topic are presented in a good number of articles including:

• Why ContentEditable is Terrible — Medium Engineering;
• ContentEditable — The Good, the Bad and the Ugly;
• Fixing ContentEditable;
• What’s wrong with contentEditable;
• The DOM is not a Datastore.

The key conclusions are:

• the current implementation of ContentEditable is wrong, broken;
• What-You-See-Is-What-You-Get (WYSIWYG) can’t be implemented correctly;
• available APIs and their implementations such as selection, clipboard and drag-and-drop are incomplete and come with numerous bugs. Range API is complex and hard to use;
• each browser implements ContentEditable in a completely different way.

And we fully agree with all the above-mentioned conclusions.

In response to this situation, it became necessary to develop brand-new editors. Generally speaking, such editors should have a custom data model, virtual DOM, strong control over the users’ input and the content of an editable element. This custom model is the single source of truth and the key element through which all the interactions with the editor take place.

It has led to the emergence of powerful solutions such as CKEditor 5, ProseMirror, Quill, Trix, Draft.js, Slate and others.

Modern rich text editors based on the custom data model

As a result, we’ve got lots of cool ideas, solutions, and considerable efforts put into the development. But, there’s always a But.

The compatibility with many third-party solutions was almost broken. And now there are new rules to follow to interact with new editors. Every editor has its own API and related documentation. All the info is publicly available. However, third-party developers don’t always have enough time and capacity to thoroughly investigate every case.

Previously, everyone considered the Document Object Model (DOM) as the single source of truth. By using it and built-in browser APIs you were able to deal with the content of any kind of editor. Now, such an opportunity is almost missing.

When working with new editors you may face different challenges, one of them is difficulties in dealing with their markup. To work with it, you should use the editor’s API. Otherwise, your markup risks being removed or the editor’s behavior will be incorrect and unstable.

Thus, the developers of editors found the compatibility with the front-runner of spelling and grammar check solutions, Grammarly, to be broken. Different editors experienced different problems. In some of them, the markup was added and immediately removed, others demonstrated a strange behavior after the markup was added; others simply got broken as a result of this interaction.

Almost at the same time (it was 2016), issues describing this problem started emerging in GitHub repos of these editors:

• [CKEditor 5] External manipulation of the DOM (Grammarly);
• [ProseMirror] Constant re-renderering with Grammarly;
• [Quill] Work with Grammarly;
• [Trix] Issue with plugin Grammarly for Chrome;
• [Draft.js] Conflict with Grammarly chrome extension;
• [Slate] Integration with Grammarly.

All this made the developers of third-party solutions look for other ways of highlighting texts with users’ errors.

DOM, extra layer and virtual underlines

We can try to add underlines in another part of the DOM and position them relative to the original text. For this purpose, we create an additional “virtual” element next to the editable element of the editor.

We’ll be using this element for placing the blocks with underlines inside it. In doing so, we’ll be synchronizing its position with the editor after the changes in the DOM structure of the page take place. Then we just track changes in the editor. We find the text that was changed. Then, we manipulate blocks creating the effect of highlighting the text with issues.

An additional virtual layer behind a rich text editor

The idea of such an interaction appeared as a result of learning W3C materials about InputEvent. The authors of the standards address the issues concerning adding markup to editors. They suggest adding it to another place in the document instead of directly to the editor and positioning it relative to the editor’s window. These very contemplations forced us to dive deep into this question.

As a consequence of the additional layer creation, the original content of the editor remains untouched. This approach was implemented in our WProofreader product.

Highlighting text issues in a rich editor with a virtual layer

As a result, we don’t need to rely on the editors anymore. We don’t need to use their APIs and learn how they work. We have to interact with the markup of the editor only when the user wants to apply our suggestion and replace the text with a correct one.

Tricky text insertion

To achieve this goal, we could use several approaches.

We’ve already mentioned the first approach — it’s based on the standard Range API. We simply manipulate the contents of range with the code using built-in methods of this range object and replace an incorrect text with a correct one.

Editors react to these changes in a different way. The main problem lies in the wrong position of the cursor after these changes — it does not correspond to reality. As a result, the markup turns out to be incorrect as well.

The second approach is based on using the text insertion method built in a browser:

document.execCommand('insertText', false, 'text');

Spell-checkers integrated into browsers and the mechanism of mobile predictive text (autocorrection) use a similar functionality for text insertion. However, even in this case, editors sometimes behave incorrectly.

ProseMirror, Quill and Trix respond well to this and the text is inserted perfectly.

Quill editor works properly with the text replacement

In some cases, CKEditor 5 has problems with such a text insertion. There’s a GitHub issue describing this problem. To fix it, CKEditor developers provide access to their instance object and we can use their API for text insertion. With this functionality, text insertion is made flawlessly without any problems. It’s great that the developers quickly responded to this problem and helped to solve it.

Slate has problems with a correct position of cursor at the time of these changes and after them. There are GitHub issues describing this problem but it’s still unclear whether or not the developers are going to solve this problem:

• [Slate] Spellcheck not working correctly on Windows/Chrome latest
• [Slate] Spell-check is editing on wrong range in markdown example

Draft.js responds to such an insertion in a bit strange way: the text is partially duplicated and the editor starts behaving improperly. You may notice errors in the console. The further interaction is impossible. It’s weird but there’s no reaction from the developers to this problem:

• [Draft.js] Native browser spellcheck causes errors after replacement in Firefox and Safari
• [Draft.js] Android keyboard suggestions are not properly inserted

Problems with a native browser spell-checker in Draft.js

There’s the third way of inserting the text into the editor. It seems like a way round, a hack. You can call a fake Delete keydown event and only after that insert the text. In doing so, the editor performs all the necessary actions to delete the text correctly and we just have to insert a new text into a predefined place. But it’s risky and insecure in the long run. We have to continually monitor how editors behave in this case and we should always be ready to invent new workarounds.

The ideal situation would be if editors allow making text insertion in the first two methods (Range API and document.execCommand). The developers of CKEditor 5 intend to do this. ProseMirror, Quill and Trix already support these methods. We hope other developers will support this idea and make the necessary adjustments.

Otherwise, we face a vicious circle just like with cross-browser issues when developers of browsers put bugs and incompatibility issues on a waiting list or choose to ignore such problems. But now it’s us who need to invent a roundabout way to provide a normal interaction. This is because editors support browser APIs in varying degrees.

The developers of modern editors are tired of fighting with browser incompatibilities. They’re exhausted to use imperfect built-in APIs. They chose to put aside these problems, get more control and provide users with more opportunities. They’re absolutely right, however, they don’t have to ignore the problems third-party developers have faced — the cross-editor incompatibilities turned into another headache for us.

Pills for a headache

There are several scenarios of solving these problems.

The ideal scenario is described above and it implies fixing the way of how editors respond to the text insertion using standard browser methods.

Also, it’s possible that editors may enable users to work with their instance objects and use their APIs, however, they should make these APIs similar, cross-editor as does CKEditor 5. It seems to be the most complex option and the developers of editors are unlikely to choose it, however, we added it to the list of possible scenarios.

The providers of spell-check and similar solutions may also help other developers using these editors alongside their spell-check tools. Everything rarely goes smoothly: in some cases you need to have a greater control over the text insertion or reject it if necessary. For this purpose, we call the custom event on the editable element (customInsertText). This event is called prior to the text insertion and it has additional data related to the insertion.

Developers can add listeners to this event, extract useful information from it and, if necessary, reject the text insertion. And they can either do this using built-in mechanisms of a certain editor (native APIs) or don’t do this at all, if the situation so requires.

We’ve already implemented this mechanism in our application and we’ve tested it on the Draft.js editor. You can read a short description of this approach in the comments in the Draft.js editor GitHub repo.

Forced evolution to the bright future

To support modern editors, we had to switch from the standard straightforward way to the alternative approach that implies using a virtual layer. As a result, we applied this approach to lots of other editable elements. It turns out to be more efficient than the former one.

It would be absolutely great if the developers of modern editors don’t stand by this process, support it, and solve existing incompatibility issues. In this case, we’ll be able to wait for the bright future together: when all the browsers will be supporting the specification of Input Events Level 2 and editors will switch to this type of events, there will be an opportunity to call beforeinput event with insertText type and all the necessary data for insertion, and we won’t bother about various types of compatibilities.