An understanding of the Kraft inequality without a prefix code

Http://blog.csdn.net/mike190267481/article/details/6941682
In the course of reading Gallager's digital communication principles, the clarft inequality, which is very familiar with information theory, is roughly as follows:

Gallager gives us a very good way to understand the code length. It maps the code length to a specific real number. Specifically, it first expresses the code word as an extension of 2 in the binary number system. For example, the codeword 011 is first expressed as 0.011 and then expanded at the bottom of 2 to 1/4 + 1/8 = 3/8.

With the above preparations, we can grasp the premise of the Theorem Without a prefix. Therefore, the 011 and 011110, 01111110, and 011111110 codes certainly do not meet the preceding conditions, it means that if we select 011 as a symbol code word, then 011110, 0111110, 011111110 ....... and so on.

How can we understand it in an intuitive way? Actually, it's very simple! For a specific code word, once we select it, it actually overwrites a range, which is a set of all other code words prefixed with this code word. The definition interval is as follows:

The Interval Length is 2-l power.

Proof: because all the code words have no prefix code, the length of an interval is excluded from each selection of a code word. For example, if we select code 1, it is expressed as 0.1, expansion at the bottom of 2 is 1/2, and the length is 1/2, that is, the interval [0.5, 1). At the same time, it means that we cannot choose these codes in the interval [0.5, 1, because all the corresponding products are prefixed with 0.1, for example, 0.11 is 1/2 + 1/4 = 3/4 in the interval.

Then we select 01 as the code word, and its range is [0.25, 0.5). Then we select 00, and its range is [0, 0.25). All the conditions are met.

Conclusion: the essence of no prefix code is that in the range [], intervals are constantly divided. The interval size is determined by the length of the codeword. The only condition that these intervals must meet is that they do not overlap, no matter how many code words you have, the length and maximum value of these segments are actually 1 in unit length.

Gallager gives us a very good way to understand the code length. It maps the code length to a specific real number. Specifically, it first expresses the code word as an extension of 2 in the binary number system. For example, the codeword 011 is first expressed as 0.011 and then expanded at the bottom of 2 to 1/4 + 1/8 = 3/8.

With the above preparations, we can grasp the premise of the Theorem Without a prefix. Therefore, the 011 and 011110, 01111110, and 011111110 codes certainly do not meet the preceding conditions, it means that if we select 011 as a symbol code word, then 011110, 0111110, 011111110 ....... and so on.

How can we understand it in an intuitive way? Actually, it's very simple! For a specific code word, once we select it, it actually overwrites a range, which is a set of all other code words prefixed with this code word. The definition interval is as follows:

The Interval Length is 2-l power.

Proof: because all the code words have no prefix code, the length of an interval is excluded from each selection of a code word. For example, if we select code 1, it is expressed as 0.1, expansion at the bottom of 2 is 1/2, and the length is 1/2, that is, the interval [0.5, 1). At the same time, it means that we cannot choose these codes in the interval [0.5, 1, because all the corresponding products are prefixed with 0.1, for example, 0.11 is 1/2 + 1/4 = 3/4 in the interval.

Then we select 01 as the code word, and its range is [0.25, 0.5). Then we select 00, and its range is [0, 0.25). All the conditions are met.

Conclusion: the essence of no prefix code is that in the range [], intervals are constantly divided. The interval size is determined by the length of the codeword. The only condition that these intervals must meet is that they do not overlap, no matter how many code words you have, the length and maximum value of these segments are actually 1 in unit length.