Information Theory Learning

Information Theory and coding

Information Theory and coding

1. Introduction

Based on probability theory, pay attention to concept understanding, not excessive pursuit of mathematical deduction

Founder of Information Theory (C.e.shannon) Shannon, American scientist, published the Mathematical Theory of communication

2. Information Signal Message

The communication system transmits the signal, the signal is the carrier of the message, the unknown component in the message is the information.

3. The smaller the probability of an event occurring, the greater the amount of information in the event (the more unknown part).

The greater the randomness of the message, the greater the amount of information that is contained.

The greater the dependency between the two messages, the greater the amount of mutual information.

4. Modulation mode has amplitude keying ask, frequency shift keying FSK, Phase shift keyed PSK

5 Information Theory Research object and content  

Research object: Message transmission system is information transmission system, communication system.

Objective: To improve the reliability and effectiveness of communication systems.

Note: Reliable: The signal does not work: fast, less resources

6 Measurement of information

6.1 Self-information and conditional self-information

1. uncertainty is related to the state number of the source probability space and its probability distribution.

2. Maximum uncertainty of equal-distribution

3. The self-information of random events is defined as the negative value of the logarithm ofthe probability of occurrence of the event 2016-05-29.

6.2

Information Theory Learning