Frequency analysis explained
For centuries the best way to hide a message was to swap every letter of the alphabet for another one. The number of possible keys is astronomical, so the cipher looked unbreakable. Yet a ninth-century scholar in Baghdad had already found its weakness: a language uses some letters far more often than others, and no amount of swapping hides that. Frequency analysis turned codebreaking from guesswork into a method, and it is still the first thing anyone tries on a new cipher.
The substitution cipher
A simple substitution cipher replaces each letter of the alphabet by another, according to a fixed table: perhaps A becomes Q, B becomes W and so on, with every letter used once. The table is the key. Unlike the Caesar cipher, which only shifts the alphabet and has 25 keys, the substitution cipher can use any order of the letters: 26! keys, about 4 × 10²⁶. Even a computer testing a billion keys a second would need more than ten billion years to try them all.
Al-Kindi's discovery
Around the year 850 the Arab philosopher Abu Yusuf al-Kindi wrote a treatise on deciphering secret messages, the oldest known description of frequency analysis. His advice was to take a long text in the same language, count how often each letter occurs, then count the letters of the secret message and match them up: the most frequent with the most frequent, the next with the next. Scholars studying the Quran had counted letters and words for generations, which may be how the idea arose.
Letters, pairs and words
In English about 12.7% of letters are E, 9% are T and 8% are A, while Z appears once in more than a thousand letters. Single counts only point in the right direction, especially in short texts, so codebreakers use every pattern they can find. A one-letter word is A or I. The most common pairs are TH, HE, IN and ER, and the most common doubled letters LL, EE, SS and OO. Q is almost always followed by U. A frequent three-letter word is probably THE, and once T, H and E are known, words such as THAT and THERE fall into place. Every language has its own fingerprint: E is even more dominant in German and Dutch, while A leads in Portuguese.
A queen's fatal cipher
In 1586 Mary, Queen of Scots, imprisoned in England, exchanged encrypted letters with Anthony Babington, who was plotting to free her and kill Queen Elizabeth I. The letters were smuggled out in beer barrels, but they passed through the hands of Elizabeth's spymaster Francis Walsingham, whose codebreaker Thomas Phelippes broke the cipher, a substitution with some extra symbols for common words, by frequency analysis. The decrypted letters proved Mary's part in the plot, and she was executed in February 1587.
Hiding the frequencies
Cipher makers fought back. Homophonic ciphers gave common letters several symbols each, so that every symbol occurred about equally often. Nulls, meaningless symbols, were scattered through the text to confuse the counts. The real answer came with polyalphabetic ciphers such as the Vigenère cipher of the 16th century, which switch between several alphabets as they go, so that one plain letter becomes many different cipher letters. It held out until Charles Babbage and Friedrich Kasiski showed in the 19th century how to find the length of the key and then apply frequency analysis to each alphabet separately.
How computers solve cryptograms
A program cannot try every key either, but it can climb. It starts from a guess, scores the result by how much it looks like the language, usually by how common its groups of two to four letters are, and then swaps pairs of letters in the key, keeping each swap that raises the score. When no swap helps, it shakes the key and starts climbing again, keeping the best result. This method, called hill climbing, solves a substitution cipher of a few hundred letters in a fraction of a second, and variants of it have cracked historical ciphers that resisted people for centuries.
Frequency analysis today
Modern ciphers such as AES are designed so that the ciphertext looks completely random: every byte value occurs equally often, and the same message encrypts differently every time. Frequency analysis has no grip on them. The idea lives on all the same: statistics of language help to recognise when a decryption is right, to break the ciphers of puzzles and escape rooms, and to read old encrypted letters found in archives. In 2023, letters of Mary, Queen of Scots, that had been lost for centuries were decrypted with computer help.
Explore it
Open the frequency analysis tool for an encrypted paragraph in English, German, French, Spanish, Italian, Portuguese or Dutch. Compare the letter counts with the language in the chart, use the clues of pairs, doubled letters and short words, and fill in the key letter by letter. Hint and Check help when you are stuck, and the solver shows how hill climbing finishes the job.
Frequently asked questions
Why is E the most common letter in English?
Because it appears in many of the most frequent words, such as THE, BE, HE, WE and ARE, and in common endings like -ED and -ES. Most European languages share this: E leads in French, German, Dutch and Spanish as well.
Can frequency analysis break any cipher?
No. It breaks ciphers in which each plain letter is always replaced by the same symbol. Ciphers that change the substitution as they go, or modern ciphers that work on bits rather than letters, need other methods or cannot be broken at all.
What is a cryptogram puzzle?
A quotation or sentence encrypted with a simple substitution, printed for readers to solve by hand. Cryptograms have appeared in newspapers since the 19th century and are solved exactly the way the tool shows: start with the short words and the most common letters.
How did the decryption of Mary, Queen of Scots's lost letters work?
In 2023 George Lasry, Norbert Biermann and Satoshi Tomokiyo identified 57 encrypted letters in a French library as hers. The cipher used nearly 200 symbols, so they combined computer hill climbing, which suggested the likely meaning of many symbols, with careful manual work on the rest.
Ready to try it?
Open PictorX in your browser — no install, no credit card required to start.
Open the frequency analysis tool