Simple Research on arithmetic coding

Arithmetic CodingIt is a lossless data compression method and an entropy encoding method. Unlike other entropy coding methods, other entropy coding methods usually divide input messages into symbols and encode each symbol, the arithmetic encoding directly encodes the entire input message into a number, which is equal to or equal to 0.0.N<1.0) decimal n.

Directory [Hide] 1 Arithmetic Coding Principle 2. Precision and Regularization 3 connections between arithmetic coding and other compression methods 3.1 Harman Encoding 3.2 interval Encoding 4. American patent on arithmetic coding 5 References 6 external links

[Edit] How arithmetic coding works

Given the symbol set and symbol probability, arithmetic coding can give near-optimal encoding results. The computation algorithm that uses arithmetic encoding usually estimates the probability of the input symbol before encoding. The more accurate this estimation is, the closer the encoding result is to the optimal result.

Example: Observe a simple signal source. The statistical model is as follows:

• 60% chance of occurrence of symbol neutral
• 20% chance of occurrence of symbolic positive
• 10% chance of occurrence of symbol negative
• 10% lead to the symbol data Terminator.(This symbol indicates that the signal source is 'internal abort', which is common during data compression. When this is the first and only time you see this symbol, the decoder will know that the entire signal stream has been decoded .)

Arithmetic Coding can be used to process more than four types of symbols. More complex cases can also be processed, includingAdvanced. The so-called high-order condition means that the probability of the current symbol to appear is affected by the previous symbol. At this time, the previous symbol is also calledContext. For example, when encoding an English documentQOrQLettersUThe probability of appearance is greatly increased. This model can also be used.AdaptiveThe change, that is, the estimation of Probability Distribution in a certain context is updated automatically with the symbol in each such context, which is more in line with the actual probability distribution. The decoder must use the same model regardless of the model used by the encoder.

A simple example is how to encode a symbolic sequence: assume there is a sequence consisting of A, B, and C symbols with equal opportunity. 2 bits is a waste of representation of a symbol for simple grouping encoding: one of the symbols does not need to be passed (As can be seen below ). Therefore, this sequence can be expressed by rational numbers between 0 and 2 in a three-digit system, and each digit represents a symbol. For example, the sequence "abbcab" can be changed to 0.011201 (base3) (0 is A, 1 is B, 2 is C ). Encode the number with a fixed-point binary number so that there is sufficient precision in restoring the representation of the symbol, such as 0.001011001 (base2)-only 9 bits are used, less than the simple grouping encoding (1-9/12) x100 % = 25%. This is feasible for long sequences because there are efficient and appropriate algorithms to accurately convert arbitrary numbers.

Each step of the coding process is the same except the last step. The encoder usually needs to consider the following three types of data:

• The next symbol to be encoded
• Current interval (before the first symbol is compiled, this interval is [0, 1), but will change each encoding interval)
• Probability Distribution of each symbol that may appear in this step in the model (as mentioned above, the probability of each step is not necessarily the same in a higher-order or adaptive model)

Encode it to divide the current interval into several subintervals. The length of each subinterval is proportional to the probability of the corresponding symbol in the current context. The subinterval corresponding to the symbol to be encoded is the initial interval in the next encoding.

Example: For the above 4-symbol model:

• The neutral interval is [0, 0.6)
• The positive interval is [0.6, 0.8)
• The negative value ranges from 0.8 to 0.9)
• The interval corresponding to the data Terminator is [0.9, 1)

When all symbols are encoded, the final result interval uniquely identifies the encoded symbol sequence. Anyone who uses this interval and the model parameters can decode and reconstruct the symbolic sequence.

In fact, we do not need to transmit the final result range. In fact, we only need to transmit a decimal point in the range. In practice, you only need to transmit enough digits (regardless of the number of digits) to ensure that all decimals starting with these digits are in the result range.

Example: Decode a section of information encoded using the 4-symbol model mentioned above. The encoding result is 0.538 (for ease of understanding, here we use decimal instead of binary; we also assume that the number of digits of the result we get is exactly enough for our decoding. These two questions will be discussed below ).

As the encoder does, we start from the range [0, 1) and use the same model to divide it into four subintervals necessary for the encoder. The score 0.538 falls in the subinterval [0, 0.6) Where neutral is located. This shows us that the first symbol read by the encoder must be neutral, in this way, we can remember it as the first symbol of the message.

Then we divide the interval [0, 0.6) into subintervals:

• The neutral interval is [0, 0.36)-- 0.6 of [0, 60%)
• The positive interval is [0.36, 0.48)-- 0.6 of [0, 20%)
• The negative value range is [0.48, 0.54)-- 0.6 of [0, 10%)
• The range of the Data Terminator is [0.54, 0.6 ).-- 0.6 of [0, 10%)

Our score. 538 is in the range [0.48, 0.54). Therefore, the second symbol of the message must be negative.

We divide the current interval into subintervals again:

• The neutral interval is [0.48, 0.516)
• The positive interval is [0.516, 0.528)
• The negative value range is [0.528, 0.534)
• The range of the Data Terminator is [0.534, 0.540 ).

Our score. 538 falls in the end-of-data range. Therefore, this must be the next symbol. Because it is also an internal end symbol, this means that the encoding has ended. (If the data stream does not end internally, we need to know where the data stream ends from other channels -- otherwise we will continue decoding, incorrectly read data not generated by actual encoding .)

The same message can be encoded with the same short score for example. 534,. 535,. 536,. 537, or. 539. This indicates that using decimal instead of binary will reduce the efficiency. This is correct because the content of the three-digit decimal data can be expressed in about 9.966 bits, and the same information can be expressed as binary scores. 10001010 (equivalent to 0.5390625), which only requires 8 digits. This is slightly greater than the information content itself or the information entropy of the message, probably the 0.6%-bit information entropy with a probability of 7.361. (Note that the last 0 must be expressed in the binary score, otherwise the message will become uncertain .)

[Edit] precision and Normalization

The interpretation of arithmetic coding is simplified. In particular, it seems that the arithmetic code first calculates the total score of the last node using a numerical value with infinite precision, and then converts the score to the final form at the end of the encoding. Many arithmetic encoders use priority-precision numeric calculations instead of Simulating Infinite precision as much as possible, because they know that the decoder can match and round the calculated score to the corresponding value. An example shows that a model should divide the interval [0, 1] into three parts and use 8-bit precision. Note that the accuracy is known, and the range of binary values that can be used is also known.

Symbol	Probability (expressed by score)	Interval between 8-bit precision reduction (expressed by scores)	Interval between 8-bit precision reduction (expressed in binary)	Binary range
A	1/3	[0, 85/256)	[0.00000000, 0.01010101)	00000000-01010100
B	1/3	[85/256, 171/256)	[0.01010101, 0.10101011)	01010101-10101010
C	1/3	[171/256, 1)	[0.10101011, 1.00000000)	10101011-11111111

ARe-NormalizationThe limited precision is no longer the limit on the number of characters that can be encoded. When all values in the specified range are shared with specific numbers, those numbers are sent to the output data. DespiteYesThe precision of processing many digits. The number of digits used for encoding is smaller than their precision, so that the existing data is moved left and new data bits are added to the Right to expand the available data range as much as possible. Note that this result appears in two of the preceding three examples.

Symbol	Probability	Range	Data bit that can be output	Normalized range
A	1/3	00000000-01010100	0	00000000-10101001
B	1/3	01010101-10101010	None	00101010-11010101
C	1/3	10101011-11111111	1	01010110-11111111

[Edit] contact for arithmetic encoding and other compression methods [edit] Harman Encoding

There is a great similarity between arithmetic coding and Harman encoding-in fact, Harman encoding is only a special case of arithmetic coding-but because arithmetic coding translates the entire message into a representation baseBInstead of translating each symbol in the message into a seriesBBase number, which is generally more optimal than the Harman encoding.

[Edit] interval Encoding

Arithmetic Coding has a deep similarity with interval encoding. They are so similar that they are generally considered to have the same performance, if there is any difference, it's just that the interval encoding lags behind a few bits. Unlike arithmetic coding, interval encoding is generally considered not covered by any company's patents.

The interval encoding principle is as follows: it does not start from [0, 1] Like arithmetic encoding and divides each character into different cells based on the probability of occurrence, it starts from a large non-negative integer range such as 000,000,000,000 to 999,999,999,999, and is divided into small subintervals Based on the probability of each character. When the number at the beginning of the final result appears as small as a certain number in the subinterval, those numbers can be moved left out of the entire operation, and replace it with the number on the right-each time a shift occurs, it is roughly equivalent to a regression amplification (usually active multiplication) in the initial interval ).

[Edit] American patent on arithmetic coding

Many different methods used for arithmetic coding are protected by American patents. Some patents are critical to implementing the arithmetic coding algorithms defined in some international standards. In this case, these patents usually followReasonable and non-discriminatory(RAND) Use of authorization protocols (at least as a strategy of the Standards Board ). In some famous cases (including some patents involving IBM), these authorizations are free of charge, while in other cases, a certain amount of authorization fee is charged. The authorization agreement under the Rand clause may not necessarily satisfy all users who intend to use the technology, because this fee is "reasonable" for a company that intends to produce software with ownership ", it is unreasonable for free software and open-source software projects.

IBM is a famous company that has made a lot of pioneering work in the field of arithmetic coding and has many patents. Some analysts feel that there is no practical and effective arithmetic code that can be achieved without breaking the patents of IBM and other companies, but is a continuous urban legend in the Data Compression industry. (especially when you see that the valid arithmetic code has been used for a long time before the initial patent expires ). However, because the patent law does not provide a "clear line" test, it is always worrying that the Court will find a special application that violates the patent, and with the detailed review of the scope of patents, it will find that a poor ruling will cause great losses. patent protection for these technologies, however, has a blocking effect on their applications. At least one important compression software, Bzip2, deliberately stopped using arithmetic coding and switched to Huffman encoding out of patent concerns.

The American patents for arithmetic coding are listed below.

• Patent 4,122,440-(IBM) Submission date March 4, 1977, approval date Oct 24,197 8 (expired now)
• Patent 4,286,256-(IBM) Approval date Aug 25,198 1 (probably expired)
• Patent 4,467,317-(IBM) Approval date Aug 21,198 4 (probably expired)
• Patent 4,652,856-(IBM) Approval date Feb 4, 1986 (probably expired)
• Patent 4,891,643-(IBM) submission time:, approval date:
• Patent 4,905,297-(IBM) Approval date Feb 27,199 0
• Patent 4,933,883-(IBM) Approval date Jun 12,199 0
• Patent 4,935,882-(IBM) Approval date Jun 19,199 0
• Patent 4,989,000 -(???) Submission time: 1989/06/19; approval date: 1991/01/29
• Patent 5,099,440
• Patch 5,272,478-(Ricoh)

Note: This list does not cover all patents. For more patent information, see the link below. [1]

Arithmetic Coding patents may exist in the judicial field of other countries. For details, refer to the software patent discussion on software patent in various parts of the world.