Artificial Intelligence, Copywriting, and Creativity

The Dummy Guide to ‘Perplexity’ and ‘Burstiness’ in AI-generated content

Understanding Language Models: A Simplified Guide

AI is fast becoming a part of our everyday lives, and with that comes the need for general knowledge of its inner workings. So here’s a simplified explanation of two main concepts: perplexity and burstiness. Together, these two metrics can help us better understand how large language models write, and how we are able to detect AI-generated content.

Perplexity

Perplexity is a measure used to evaluate the performance of language models. It refers to how well the model is able to predict the next word in a sequence of words. As you’ll probably know by now, AI-generated text is procedurally generated; i.e. word-by-word. AI selects the next probable word in a sentence from a K-number of weighted options in the sample.

Machine Learning: When generating text, an a text transformer will ask, “What comes next?”

Perplexity is based on the concept of entropy, which is the amount of chaos or randomness in a system. So a lower perplexity score indicates that the language model is better at calculating the next word that is likely to occur in a given sequence, while a higher perplexity score indicates that the model is less accurate. Basically, the…