Entropy
Randomness
Information content
Different symbols
  • Given the previous symbols
  • ZIP (deflate)
  • Bits per symbol, against the number of previous symbols known

Character Count Probability Surprise (bits) Share of the entropy
Count by

More information about entropy

Entropy calculator

How much information is there in a text? Claude Shannon's answer, entropy, counts it in bits: a symbol that is hard to predict tells you a lot, one you could have guessed tells you little. Paste any text or pick an example and see its entropy per character, word or byte, how close it comes to pure randomness, and how much it could be compressed. Then watch the information per symbol drop as the symbols before it are taken into account, the trick that every compression program uses. Type or paste a text, or pick an example. The app counts how often each symbol occurs and computes the Shannon entropy: the average number of bits of information per symbol. Count characters, words or bytes. The table gives each symbol's surprise in bits, and the second chart shows how much less information each symbol carries when the symbols before it are known, next to what ZIP compression achieves.

How it works

  1. Type or paste a text, or choose an example: a paragraph, random letters, bits, digits or DNA, or a repeating pattern.
  2. Choose whether to count characters, words or bytes, and whether upper and lower case count as the same.
  3. Read the entropy, randomness and information content in the tiles, each symbol's surprise in the table, and how context and ZIP compression lower the bits per symbol in the second chart.

What entropy measures

A symbol that occurs with probability p carries −log₂ p bits of information: a fair coin toss gives 1 bit, a symbol with a 1 in 8 chance gives 3 bits. Entropy is the average of that surprise over all symbols of the text, H = −Σ p log₂ p. It is largest when every symbol is equally common, log₂ n bits for n different symbols, and zero when the text repeats one symbol. Multiplied by the length of the text it gives the fewest bits a code could use if it encoded every symbol on its own, which is why entropy sets the limit for compression.

Why context lowers it

Counted one by one, the letters of English carry about 4 bits each. But a reader can often guess the next letter: after Q comes U, after TH often E. The conditional entropy, the information in the next symbol when the previous ones are known, falls as more context is used, and Shannon estimated that a fluent reader brings English down to about 1 bit per letter. Compression programs such as ZIP exploit the same redundancy. Random text has none: its entropy stays at log₂ n however much context you use, and it cannot be compressed.

Frequently asked questions

What is a bit of information?

The information you get from the answer to a yes-or-no question whose two answers were equally likely, such as a fair coin toss. Two such questions give 2 bits and four possibilities, three give 3 bits and eight possibilities: n bits single out one of 2ⁿ equally likely options.

Why is the randomness never quite 100% for real text?

Because the symbols of a language are not equally common. The randomness tile divides the entropy by its largest possible value, log₂ of the number of different symbols, which is only reached when every symbol occurs equally often. Random letters come close to 100%; ordinary text, with its many E's and few Z's, typically reaches 80 to 90% counted letter by letter.

Why do the longer contexts look so predictable for a short text?

With a short text most combinations of a few symbols occur only once, so whatever follows them looks certain. The estimate is then far too low. The chart dashes that part of the curve; paste a longer text, thousands of characters, for a fair picture.

Is this the same as password entropy?

Related but not the same. Password strength counts the possibilities a generator could have produced, length × log₂ of the alphabet, which is what an attacker must search. This calculator measures the statistics of one given text. A password like aaaaaa has low entropy here, and it is weak either way.

Is my text sent anywhere?

No. Everything, including the ZIP comparison, is computed in your browser, and your text stays in this browser's local storage.

Ready to try it?

Open PictorX in your browser — no install, no credit card required to start.

Go to Editor