Some Conclusions

• Example: The binary sequence
  1111001001100111000010111011000
  can be decoded as
  wethepeople
• To encode all occurrences of the 7 characters we have considered requires
  $2 \cdot 3320 + 3 \cdot 1458 + 3 \cdot 1067 + 3 \cdot 1749 + 4 \cdot 547 + 2 \cdot 2474 + 4 \cdot 266 = 27662 bits.$
• Hence, we require on average 2.54 bits per character.
• Information theorists have shown that at least
  $H = -p('e')\log_2p('e')-\ldots-p('w')\log_2p('w')=2.49bits$are required on average for representing the 7 characters in our example.
• The quantity H is called the entropy and is a fundamental limit that no encoding method can ever exceed.
• Huffman coding achieves efficiencies very close to the entropy H.
• The entropy of the entire constitution is 4.1 bits per character.
• ASCII encoding uses 8 bits per character.
• Hence, we can expect approximately 50% compression (4.1/8) when we use Huffman coding on the entire constitution.