MPEG2 TS Summary

1. It should be said that the real understanding of TS, or to see a friend recommended "digital television business information and its code" after the book, MPEG2 TS and digital television is closely inseparable, it is worth summing up some of the relationship.

Iso/iec-13818-1: System part; Iso/iec-13818-2: Video; Iso/iec-13818-3: Audio; ISO/IEC-13818-4: Conformance testing; ISO/IEC-13818-5: Software part; ISO/IEC-13818-6: Digital Storage media command and control; ISO/IEC-13818-7: Advanced Audio coding; Iso/iec-13818-8: System decoding real-time interface;

2. MPEG2 system tasks include:

1. A protocol that provides for the transmission of data by packet; 2. The protocol that specifies the synchronization of data flow between two ends; 3. Provide multiplexing and multiplexing protocols for multiple data streams; 4. Provide the protocol for data stream encryption. Storing and transmitting data streams in package form is a key point of MPEG2 system.

3. PES:

ES is a data stream that comes directly from the encoder , which can be an encoded video stream, a stream of audio data , or a collectively referred to as his encoded data stream .

The es stream is converted into a PES package after it has been through the PEs packager . The PES package consists of Baotou and payload , and the specific format is summarized as follows:

You can see that Pts/dts is in the PES package , and these two parameters are the key to resolving the video-Audio synchronization display , which prevents overflow or underflow of the decoder input cache .

PTS indicates the time at which the display unit appears on the system target decoder (Std:system target decoder).

DTS represents the moment when all bytes of the access unit are removed from the ES decoding buffer of Std.

Each I, P, B-Frame header has a pts and DTS, but pts and DTS are the same for B-frames, without having to mark B-Frame DTS.

For I-frames and P-frames , the display must be stored in the video decoder's re-order buffer, after a delay (reorder) to display, be sure to mark Pts and DTS.

4. PS Stream:

As described in the previous section, ES first needs to be packaged into a PES streaming package, and PES is packaged as a PS or TS package for storage or transfer. Each of its ES contains only the encoded data stream of one source, so each PES also contains only the data stream relative to the source.
For PS streams, each PES header contains pts and DTS, stream identification codes, which are used to differentiate ES in different properties.

The PES package is then re-used into PS package via PS multiplexer. The fact is that the PES package is decomposed into a smaller PS package.

At the time of decoding, the multiplexer decomposes PS into a PES package, unpacking and then splitting the PES package into ES of video and audio, and then decoding it into the respective decoder.

One problem is: How to ensure the synchronization of audio and video when each es is decoded? In addition to working with PTS and DTS, an important parameter is SCR (System clock Reference).

Both Pts,dts and SCR are generated by the STC (System time clock) during encoding.

When decoding, the STC regenerates and, using the PLL (Pll-phase lock Loop), compares the local SCR phase with the instantaneous SCR phase lock of the input to determine if the decoding process is synchronous and, if not synchronized, adjusts the local clock frequency of 27MHz with this instantaneous SCR. Finally, Pts,dts and SCR work together to solve the problem of video-audio synchronization playback. The PS format excerpt is as follows:

PS package length is longer and variable, mainly used in the error-prone environment, because the longer the more difficult synchronization, and in the case of packet loss, reorganization is also more difficult. Therefore, PS is suitable for the editing of program information and the application of local content application.

5. TS Stream:
TS Stream is also a combination of one or more PES, they can have the same time base, or can be different. The basic reuse idea is that multiple PES with the same time benchmark are now re-used for program reuse, and then multiplexing each PS with independent time benchmarks to produce TS eventually.
The TS package consists of 2 parts of Baotou and packet data, of which Baotou can also include extended self-applicability zones. Baotou length accounted for 4bytes, since the use of the zone and the package data accounted for 184bytes, the entire TS package length equivalent to 4 ATM packets long. The packet header of the TS package consists of 8 parts: synchronization byte, transmission error indicator, Payload unit start indicator, transmission priority, packet identification (pid-packet identification), transmission scrambling control, adaptive zone control, and continuous counter, as shown in the excerpt.

Among them, can use the automatic correlation characteristic of the synchronous byte bit string, detect the packet limit in the data stream, set up the packet synchronization, the transmission error indicator, the error correction decoder can be used to indicate the error of 1bit, but can not be corrected; The starting indicator of the payload unit indicating whether the packet has a definite starting information Transmission priority is to assign priority to the TS packet, the PID value is determined by the user, the decoder according to the PID will never be the TS with ES from the TS package to reconstruct the original ES; transmission scrambling control can indicate whether the packet content is scrambling, but the Baotou and adaptive zone is never disturbed; adaptive zone control, with 2 Bit indicates whether there is an adaptive zone, i.e. (01) indicates that there is no adaptive zone for useful information, (10) indicates that no useful information has an adaptive region, (11) indicates that there is a useful information there is an adaptive zone, (00) is undefined; the continuous counter can count the PID packet delivery order, according to the counter readings, The receiver can determine if there is a packet loss and packet delivery sequence error. Obviously, Baotou has synchronization, identification, error detection and encryption function for TS packets.

The TS Packet Adaptive Zone consists of an adaptive warden, various flag indicators, information related to the insertion mark, and 4 parts of the fill data. The symbol part consists of 8 parts, such as discontinuous indicator, random access indicator, ES optimization indicator, PCR mark, contact sign, transmission special data mark, original PCR mark, Adaptive zone extension flag. It is important to mark part of the PCR field, which can provide synchronization data to the 27MHz clock of the codec. The process is to determine if the decoding process is synchronized and, if not synchronized, the clock frequency is adjusted using the PLL, which is used to compare the local PCR phase with the instantaneous PCR phase lock of the input during decoding. Because, the digital image uses the complex and different compression coding algorithm, causes each image the data to be different, causes the clock information to obtain directly from the compression encoded image data the beginning part to be impossible. To do this, some, but not all, TS packages are selected for adaptive zones to transmit timing information. Therefore, the adaptive zone of the selected TS package can be used to determine the control bit and important control information of the packet information. The adaptive zone does not need to accompany each package to send, how much is determined by the transport-specific time-scale parameters of the selected TS package. The random access indicator and the contact sign in the flag provide random entry points for randomly entering the data stream of the I-frame compression, as well as facilitating the insertion of local programs. The padding data in the adaptive zone is due to the fact that the PES packet length cannot be converted to an integer multiple of the TS package, and the last TS Packet retains a small amount of usable capacity, filled by padding bytes, which prevents the buffer from underflow and keeps the total bitrate constant.
The basic format of MPEG2 TS is summarized in section 3 above, including Pes,ps and TS, as well as descriptions of related fields. Then, as a transport stream, TS packages/re-uses the content to turn its media content into TS transmission and eventually decodes it on the decoding side. In a nutshell, TS is a protocol stack for a transport layer that can host a variety of content transmissions, such as mpeg,wmv,h264, or even IP, so how is the transport specification defined? This is what PSI (program-specific information) is going to do.
The PSI consists of four tables: Pat,pmt,cat and NIT, each of which describes the transmission structure of all ES streams included in a TS. One of the first concepts is that TS is transmitted as a packet, and at the codec end it is necessary to identify the contents of the TS Stream with a certain package ID, for example, the PAT table exists in one or more TS packages, so it is represented by a special package ID, and different ES streams need different package IDs to identify. We have the two tables of Pat and PMT, the decoder can be based on the PID, TS from the same es from the TS package to distinguish from the decoding.
TS decoding in two steps, one, is from the PID 0 TS package, parse out the Pat table, and then from the Pat list to find the various program source PID, generally such program source is composed of several ES streams, and described in the Pmt table inside, and then through the program source of the PID, You can retrieve the PID of each ES in the PMT table. Second, the decoder is based on the PID of the ES stream in the PMT table, which differentiates the packets on the TS stream and decodes them according to different ES streams. Therefore, TS is through the program reuse and transmission multiplexing two layers, that is, when the program is re-use, added PMT, in the transmission, joined PAT. Similarly, when the program is re-used, the PMT can be obtained and pat can be obtained when the transmission solution is reused. A good overview of its ideas.

TS is multiplexing-enabled, so it can be used to transmit multi-layered programs after reuse. In the process of reuse, it should be noted that the decoding process needs to face the time reference and synchronization problems, because de-multiplexing is required to synchronize the various information, so in the reuse process, it is necessary to insert the relevant time information: PTS,DTS,PCR.
In the TS formation process, PTS and DTS are packaged in ES into PES, according to the STC reference, the clock information into the PES package, and then in PES cut into TS, then the PID and PCR information into the TS package, when the multi-channel TS re-use, The PCR of each TS will be extracted and analyzed, then the new PCR will be generated and injected into TS according to the unified STC reference, finally, since the original Pat table information is not applicable, the new Pat table needs to be regenerated and appended to the new TS stream. After this multi-layer multiplexing, the new TS stream can enter the modulation, transmission phase. Procedures can be found in:

Decoding process to face the problem is: de-multiplexing, audio-visual synchronization, decoding buffer without overflow. De-multiplexing is the separation of TS in different time series of the same channel; video-audio synchronization is performed by DTS, pts, and PCR, and PCR is the absolute time-scale for reconstructing the system's temporal benchmarks, while DTS and PTS are relative time-scales for decoding and reproducing time, and there is no overflow problem with the decoder cache. The system target decoder (STD) model must be used to implement it, the basic idea is as follows:
1. After the TS stream enters the decoder, the various ES streams are decomposed (including the PSI information flow) by the commutator, according to a certain timing relationship.
2. After the decomposition of the ES stream will enter the respective transmission buffer, after which the PES stream into their respective main memory, note that the PSI traffic will enter the system cache, and finally reach the main memory.

3. Finally, according to DTS information, the decoder extracts media or system information from each main memory, decodes it, and displays the media content according to the PTS information.
The process is described in the following:

MPEG2 TS Summary