Organizing the World of Fonts with AI

How we created a quick experiment to inspire designers.

While exploring what others are doing with machine learning, I found an image created by researcher Andrej Karpathy, who used AI to organize thousands of photos onto a single map through higher order visual recognition. The visualization not only showed how effective AI has become at organizing visual information, but it also made me wonder how it could be applied to daily challenges we face as designers.

An image from Andrej Karpathy’s explorations. The more you look at it, the more impressive it becomes.

Exploring Fonts

Choosing a font is one of the most common visual decisions a designer makes. Often, designers fall back on fonts they’ve used before or search within categories like serif, san-serif, or grotesque. These categories are a useful starting point, but even within a font category there is a wide range of aesthetic differences. Plus, even if a designer finds the perfect font, it may not fit when applied to a layout or website. Designers need an easier way to discover alternative fonts with the same aesthetic — so I decided to see if a machine learning algorithm could sort fonts by visual characteristics, and enabling designers to explore type in a new way.

A diagram showing the flow of the machine learning algorithm from font samples using the word handgloves to the assigned point in 2d space.

My AI experiment

First, I created a training set of images, one for each font. Leveraging some of IDEO’s in-house expertise, I found that type designers often use the term “handgloves” to examine fonts, so I used this term as well when generating images for the algorithm to use. This allowed each image to contain enough characters to represent the various traits of each font.

Once I had a bunch of font images, I used a convolutional neural network called VGG16 to generate for each font a list of numbers that represents what the network thought were the notable visual features of the image. Finally, in order to get x,y points, I ran these embeddings through T-SNE, a popular algorithm for taking large vectors and compressing them into a smaller space — in this case into a 2-D plane.

Early Results

At the end of the process, I had an arrangement of about 800 fonts in a 2-D space. I was pretty surprised by how well the results came out : There were very clear font clusters like a sans-serif group, and a group of cursive fonts. The algorithm also was able to cluster outliers, or fonts that were more unique and had fewer relations with others.

Once I had this map, the next challenge was to turn this arrangement into a digital tool that would be interesting and useful for designers. Luckily, we were able to gather feedback from designers at IDEO along the way!

Introducing Font Map

The final result of this exploration is Font Map, an interactive map of more than 750 fonts organized using machine learning.

This quick dive into machine learning made me even more excited about the potential to create new, intelligent systems that can aid the design process. Assuming the data is available, mapping thousands of fonts is scalable with this approach. I’m also curious to explore how popular pairings would show up on Font Map. Is there an underlying pattern exposed through visualization that can help us understand what makes a good pairing? The design community here at IDEO came up with these questions and more, creating rich discussions around the role of AI and machine learning.

I hope this inspires fellow designers to play around with machine learning and AI, and use these new capabilities in ways that are insightful, inspiring, and valuable.

Thanks to Jochen Weber, Katie Kirsch, and Lily Shen.