What Is the Type-Token Ratio?

The type-token ratio (TTR) is the simplest measure of lexical diversity: how varied the vocabulary in a text is. You divide the number of different words by the total number of words. A text that keeps repeating the same words has a low TTR; one that rarely repeats has a high TTR. It is quick to calculate and easy to understand, but it has one big flaw that every user should know about: it depends heavily on text length.

Types and tokens

• Tokens: every running word in the text, counting repeats.
• Types: each distinct word, counted once.

Worked examples

The second sentence repeats I, like and and, so its TTR is low; the third uses each word once. You can calculate TTR for any text with the lexical diversity calculator.

The length problem

Because of Zipf's law, a small set of words (the, of, and, to) keeps recurring no matter how long a text gets, while new types appear more and more slowly. As a result, TTR falls as a text grows. A 50-word paragraph might score 0.75, while a 5,000-word essay by the same writer might score 0.30 or lower. Comparing the TTR of a short email with that of a long report tells you almost nothing about vocabulary.

Researchers have proposed several fixes:

For anything beyond a quick comparison of equal-length samples, MTLD or a standardized TTR is the better choice.

Other decisions that change the number

• Word forms or lemmas? Counting run, runs and ran as one lemma lowers TTR compared with counting raw forms.
• Case and punctuation. Is The the same type as the? Is don't one token or two?
• Numbers and names. Including them can inflate diversity.

When comparing your own texts, keep these settings the same every time.

Why TTR matters for learners

Lexical diversity is one sign of a developing productive vocabulary. Learner writing often relies on a small set of high-frequency words (good, thing, very, get) used over and over. Speaking and writing assessments such as IELTS reward a range of vocabulary used accurately, and diversity measures are used in research on second-language writing and child language development.

But diversity alone is not quality. A text stuffed with synonyms to avoid repetition can be harder to read and less precise. In technical writing, repeating the exact term is often correct.

How to use TTR in practice

1. Take two samples of your writing of the same length, for example the first 300 words of an essay from three months ago and one from today.
2. Run both through the lexical diversity calculator with the same settings.
3. Look at the words you repeat most often. The word frequency counter lists them.
4. Replace weak repeats (very good, a lot of, thing) with precise words, but keep necessary repetition of key terms.

Common mistakes

• Comparing texts of different lengths with raw TTR.
• Confusing diversity with density. Lexical density measures content words, not different words.
• Chasing a high score with a thesaurus, at the cost of clarity.

Sources: Guiraud, P. (1960), Problèmes et méthodes de la statistique linguistique; Malvern, D., Richards, B., Chipere, N. & Durán, P. (2004), Lexical Diversity and Language Development; McCarthy, P. & Jarvis, S. (2010), MTLD, vocd-D, and HD-D, Behavior Research Methods.