What is Information
September 21, 1997

• The word information is often used colloquially to refer to news, messages, etc.
• Communication theorists have come up with means to measure information.
• We to relate the everyday notion of information to the theoretical concepts developed by information theorists.
• We will discover the central role played by entropy.
• We will see that the amount of information in a message is closely related to the shortest binary representation of that message.
• To measure information one could think of a variety of methods, e.g., one could count the number of characters in a message.
• However, we will see that a much better measure for the amount of information is the smallest average number of yes-or-no questions to find the contents of the message.