Information Theory Learning

Source: Internet
Author: User

Information Theory and coding

Information Theory and coding

1. Introduction

Based on probability theory, pay attention to concept understanding, not excessive pursuit of mathematical deduction

Founder of Information Theory (C.e.shannon) Shannon, American scientist, published the Mathematical Theory of communication

2. Information Signal Message

The communication system transmits the signal, the signal is the carrier of the message, the unknown component in the message is the information.

3. The smaller the probability of an event occurring, the greater the amount of information in the event (the more unknown part).

The greater the randomness of the message, the greater the amount of information that is contained.

The greater the dependency between the two messages, the greater the amount of mutual information.

4. Modulation mode has amplitude keying ask, frequency shift keying FSK, Phase shift keyed PSK

5 Information Theory Research object and content  

Research object: Message transmission system is information transmission system, communication system.

Objective: To improve the reliability and effectiveness of communication systems.

Note: Reliable: The signal does not work: fast, less resources

6 Measurement of information

6.1 Self-information and conditional self-information

1. uncertainty is related to the state number of the source probability space and its probability distribution.

2. Maximum uncertainty of equal-distribution

3. The self-information of random events is defined as the negative value of the logarithm ofthe probability of occurrence of the event 2016-05-29.

6.2

Information Theory Learning

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.