Text Rendering on the Web for Non-Latin

An Interview with Behdad Esfahbod of Google

This week, TypeThursday goes under the hood with Behdad. We discussed how web browsers render text, the need for a text shaping engine like Harfbuzz, and the shift of design tools moving to the web as a platform.

In NYC? Join us as we share work-in-progress letterform projects and grab some type swag from R-Typography.
September 22, 2016
6:00–9:30 PM
347 W 36th St #603
New York, NY 10018
RSVP for Free

TypeThursday: Behdad, thank you so much for being here for TypeThursday.

Behdad Esfahbod: Thank you for having me here.

TT: I’d love to get started, first, with learning more about you. Can you share with us some information about yourself?

BE: Okay. I’m Iranian. About ‘99/2000, I started university. I was back in Iran. And we wanted to make Persian work on the internet. That’s how I got involved in these things. Now, I’ve got to say, my father was a printer, so I have a printing background, but I became interested in computers and then I started studying computer science and then we wanted to make Persian work on the internet. So, I got involved into learning what Unicode is and then how software text rendering works. And because we could not control Internet Explorer, I ended up working on open-source software, to GNOME and KDE desktops. And then later, I moved to Canada to continue studying and I became more and more involved in the field of Open Source software text rendering. And then I finished school and started working for Red Hat. At that point, I was maintaining the text rendering infrastructure on the Linux desktop. Four years later, I joined Google to continue the same line of work, in the context of the Chrome browser. And then Android became a thing and Android needed to render text in all kinds of languages, so I became part of the text rendering team at Google and that’s what I’ve been doing for the past few years.

HarfBuzz

TT: That’s awesome. It’s really great to have someone from a computer science background who really cares about how type looks on screen. As a type designer, I appreciate that. So, I know one of the projects you were working on is called HarfBuzz. Could you share with us what that is?

BE: Sure. So my native script is the Arabic script, which is what Persian is written in. And unlike Latin, rendering Arabic is a bit more complicated than just putting individual glyphs next to each other. In Latin — Let’s step back. When computers became a thing, the way text rendering worked was you had a character, like the character F and the character I, and to render text you just put the shape for the first character and then the shape for the next character and the shape for the next character. That’s what we call a simple script. Arabic is a bit more complicated. Each character might be rendered in one of up to four different forms depending on what characters are next to it on both sides. Now when you get to scripts like the Indic scripts or Southeast Asian scripts, the logic becomes even more complicated. HarfBuzz is an implementation of text shaping. And text shaping is exactly what we define as here’s my font, here’s my unicode text, just tell me which glyphs to show and where to show them. That’s what HarfBuzz does.

Substitution and Positioning can be used in Latin…, but while in Latin they are for aesthetic reasons, in script like Arabic or Indic, they are part of the script logic and you can’t render text without these features.

Opentype Substitution and Positioning

TT: So to clarify, you’re saying that basically in fonts and scripts like Arabic, there’s a need for substitution and positioning.

BE: Correct. Correct. That’s the way that OpenType and other technologies model this, exactly. So you have the unicode character, you choose a nominal glyph for it, but then after that you run a set of rules that say if this glyph is before this one, then substitute it with this one. And then you do a round of positioning. And positioning happens, both of them, substitution and positioning, they also can be used in Latin for ligatures and mark positioning, but while in Latin they are rather for aesthetic reasons, in script like Arabic or Indic, they are part of the script logic and you can’t render text without these features.

TT: When we say substitutions and positioning, could you define what those mean?

BE: Substitution is simple. It’s the kind of rules in a font that say if you see the F glyph followed by the I glyph, then replace it with the F-I ligature glyph. And then positioning is, kerning is a special case of positioning. Mark positioning on top of bases, that’s another kind of positioning. Again, when we get to a script like Arabic, it becomes more complicated. Like the Arabic Nastaliq style of writing, also known as Cascading Arabic, the glyphs join, they have a cursive connection. And then they shift up, they are not all lined up on the baseline, so that’s the most complicated part of positioning.

TT: And those kind of considerations were produced in HarfBuzz, correct?

BE: Right. HarfBuzz’s job is to just hide that part of that logic from the software developer. So you as a type designer have to — You test your font with HarfBuzz and make sure it works. And then the software developers on Chrome or Firefox or Android or another system, they don’t need to understand this logic. They just hand it out to HarfBuzz and it does the right thing for them.

Text Shaping for Non-Latin Languages

TT: So this kind of text shaper is specific to the scripts and it sounds like you had to be proactive on your own to help create it for Arabic, for your native language. This seems to be a problem for many scripts that are non-Latin. Do you have any thoughts about that?

BE: In HarfBuzz, we actually do many more scripts. So let me step back. OpenType was designed by Adobe and Microsoft in the late 90s. And then Microsoft continued developing OpenType to support more and more scripts. They added Indic scripts in the early 2000s, and when I say Indic scripts, there’s 10 or 11 official major scripts in India itself. So Microsoft added those and then over the years since, with every new version of Windows, they added support for a couple more scripts. And HarfBuzz supports all of that. That was one of my best learning experiences was to implement the Indic shaping logic in HarfBuzz. It took a few years, like two or three years. I tried to understand how Devanagari works and how Bengali works and write that code. It was a really interesting learning experience. HarfBuzz is much more than just Arabic. I mean there’s nothing Arabic-specific about it. It’s just one of the scripts that it supports.

TT: You said it took a couple of years to wrap your head around the Devanagari, what were some of those issues or complications about it?

BE: So there’s Unicode, right? Unicode encodes the characters. And there’s a book that comes with it, it’s a thick book. But Indian unicode is just an encoding scheme. It doesn’t tell you how to render, how to show this text or shape this text. For that, Unicode says it’s not their job; It’s an implementation detail. And then there’s OpenType, which is a font format we all use these days. And OpenType script specifications is how this works, except that the documentation that is available on Microsoft website was last updated ten years ago, and it’s a rather short document that leaves out a lot of details, that is wrong in many other places. So for me to implement this, well I started by reading everything that unicode had to say and then I started by reading everything that OpenType had to say. And then I studied the previous open source implementation that existed, and tried to distill this all in a new shaping engine. And then very early on, I noticed that even then I had a very low confidence in my code, so one of the major aspects of the project was that we decided, “Okay, because there is no well-defined specification of these things and Windows has this huge market share, whatever Windows does is the standard.” I want to design HarfBuzz so font designers get the exact same shapes out of HarfBuzz that they get out of Windows. So testing against Windows was one of the major aspects of developing HarfBuzz’s Indic shaping engine. And we did that by the brute force approach. We downloaded the entire Wikipedia for every language, created a list of all the words in Wikipedia and for Devanagari, for example, that’s about a million words. And then I had my shaper and then I had Windows shaper on a Windows machine. I could shape these million words using both shaping engines and compare the results. And then I say, “Okay, in this case, I am diverging from Microsoft,” and then I go fix it. So that’s how we tried to make life much easier for font designers by making it easier for them to design a font that works the same way everywhere, or at least on Windows and on our platform.

TT: See, I think there’s a lot of type designers in the audience reading this right now who are shaking their head in mutual pain and agreement when you say the brute force method. That sounds like it’s a very painful process to have to be done to be very thorough in your thinking process and testing the text shaper.

Issues with Text Shapers

BE: I never thought it was a major pain for most type designers, because there’s the Microsoft platform, there’s Apple and then many type designers — And there’s our platform and we tried to be very close to Microsoft, so to reduce that pain, but then again many type designers also do graphic design or their clients, whoever buys their fonts, does graphic design and graphic designers use a lot of Adobe tools. So then you end up with fonts that work on Windows and work on HarfBuzz and then they don’t work on InDesign and that’s a major pain for type designers and also for us. Because at Google, we also develop or pay for fonts to be developed. We have the Noto font family, which is a pan-Unicode font family that is probably the biggest font project ever done. It’s over 100 different scripts. And we spend all this effort in creating fonts and then we get reports from users that your font doesn’t work in InDesign. So that’s a major pain for us as well. We are fully aware of that. We are trying to work to improve this ecosystem by working with other companies and partners. But in the end, there’s only so much we can do, just as Google.

TT: To clarify, is this issue specifically with InDesign? Is it because their text engine is not aligned or not compatible with HarfBuzz or Microsoft?

BE: Right. Many of the text engines are not up to par. If you look at Adobe, Adobe’s major markets are the Western market and then the Japanese market. So when you look at those segments, Adobe has best class software and you can get the best paragraph layout or OpenType feature support or vertical Japanese. But when you get to the middle, like the Middle East or like South Asia, those are not huge Adobe markets and those are excellent Microsoft markets. So Microsoft has been driving those parts of the OpenType shaping specification. But Adobe’s lagging behind a bit. Yeah.

If you are someone in Southeast Asia and you want to use your script with your computer, you don’t have to upgrade to Windows 10. Just use a modern browser, and within the browser, you’ll get the best support for your script.

TT: Because their priorities are in other areas.

BE: Right. Exactly. There’s a lot of problems with Adobe and Microsoft products. It used to be, their release cycles used to be multiple years. You have like Windows XP and then three years later Windows 7 or two years later Windows Vista. Between these releases, they did add many new script support. But the majority of Windows users are running Windows 7, but so even though Microsoft added a lot of script support since then, they are all in Windows 8 and the users that are using Windows 7 don’t get it. Same thing with Photoshop and InDesign, they are on very long release cycles. So even if they fix it now, people might not upgrade it yet or don’t want to pay for an upgrade. So that’s an old Microsoft development model. Fortunately, everyone is going to a much faster one. Windows has moved to a — Like Window’s new service, you get annual updates, without having to pay again. Or Apple has the same kind of faster, once a year at least, update model. Now with HarfBuzz, we also have the same problem, because Android is again something like once a year release cycle. But on the browser side of things, we are much more fortunate, because both Chrome and Firefox have very fast release cycles. Something like six weeks, every six weeks you get a new release. But the nicer part is you can install the latest Chrome or Firefox on your Windows 7. So if you are someone in Southeast Asia and you want to use your script with your computer, you don’t have to upgrade to Windows 10, you can just use a modern browser, and within the browser you get the best support for your script.

The future of design tools will be moving to the web.

The Web as the Ultimate Platform

TT: You shared before that HarfBuzz is used as a text shaper for Firefox and Chrome and Android. It’s very focused on the web experience. Much of typography is the web. Do you have any commentary for type designers who are interested, especially in non-Latin type design?

BE: Well, I mean, I think I’m not the only one to think that as the web as the ultimate platform. It’s the future we want to build on, because there’s a healthy ecosystem of the web standards and the CSS working group and all these. And a few competitive browsers like Edge and Firefox and Chrome and Safari, I mean really good competition. And they have a much higher momentum to move forward and add features, especially in typography and design than a traditional app on Windows or Mac has. And those platforms, you’d be reliant on app developers who then have to put their app to different platforms and many apps that designers use might be Apple only. And when you get to the web, you don’t have any of these problems. All of the sudden, you have this rich infrastructure that you can build on without even knowing much code really. The future of design tools are moving to the web. Within the next few years, you’ll probably see much more of that.

TT: This has been a great conversation. Thank you so much for your time.

BE: You’re welcome. All right, thanks for having me.

Do you live in NYC? Come meet fellow letterform lovers, share work-in-progress letterform projects, and get type swag from R-Typography. RSVP to attend.

Enjoy these interviews? Sign up to the TypeThursday mailing list to be the first to know about our next interview.

Was this article interesting to you? Click the Recommend button below