Digital TV business psi/Si Learning Series)

 

Principle of MPEG-2 System
Chapter 1 MPEG-2 Overview
What is MPEG and MPEG-2?
MPEG is short for Moving Picture Experts Group. MPEG-2 is a digital video compression specification designed by MPEG and ISO organizations. It is mainly used on DVDs and DVB. The current standard document is iso13818.
What is DVB?
DVB, short for Digtal video broadcast, is a digital TV standard used by all European countries and other countries and regions. The American ATSC standard is also implemented based on the DVB standard.
What are ts streams and PS streams?
Ts is short for transport stream, which is the "transmission stream". The data format used by DVB data broadcast is the TS code stream.
PS is short for Program Stream, which is "Program Stream". The data format used in the DVD is PS stream.
The two stream formats are different. Ts format has a strong error correction function and is suitable for transmission, while PS format is suitable for storage in media. here, we focus on the TS code stream format.
What is PID?
PID is short for packet identification, which is the Packet Identifier ". the DVB system packages different data into different data packets and marks the data type with a unique 13 BITs number. for example, pid = 0x00 indicates the Pat package in the DVB system, while pid = 0x10 indicates the NIT package of the DVB system. different programs (including video and audio) use different PIDs. For example, if we detect that the PID = 0x0120 is video data in the PMT table, all packets whose package number is 0x0120 are video data.
What is Pat?
Pat is short for program Association table, that is, "program Association table ". pat is a packet in the stream of the DVB system. The package number (PID) is 0x00. the Pat table describes the PID contained in the stream of the DVB system. It mainly describes the PID Number of the NIT table of the current stream and the number of PMT tables of different types in the current stream, information such as the channel number corresponding to each PMT table.
What is PMT?
PMT is short for the program map table, that is, the program shadow table ". the pmt pid is freely defined by the server (but does not conflict with the PID retained by the system ). this PID is described in the Pat table. For example, if there is 0 X in the Pat table, it means that all the packets whose PID is 0 x are PMT tables. different channels have different pmts. That is to say, a different PMT represents a different channel. the PMT table and pat table work together to detect all the PIDs of all programs in the DVB stream. Therefore, the digital TV search platform relies on the data of these two tables.
What is cat?
Cat is the abbreviation of conditional access table, that is, "Conditional Access Table", and PID is 0x01. cat carries the private information of the server (the CA system uses this table to decrypt the program ).
What is SDT?
SDT is short for service description table, that is, "server description table", and PID is 0x11. the SDT carries the TV station name and TV program name. the DVB receiving system receives program information from SDT tables to display and operate programs on a user-friendly interface.
What are nit and EIT?
NIT is short for network information table, that is, "network information table ".
EIT is short for event information table, that is, "environment information table ".
NIT can provide program information of the current stream or program information of the stream associated with the current stream.
EIT provides information changes for stream programs, such as information about a program that has already started or ended.
What is EPG?
EPG is short for electronic programme guides, which is the "Electronic Program Guide". This function can be used to predict the content of the program to be played in the near future, and supports Content-based retrieval.
What is teletext?
Teletext is what we often say "text TV", of course, in the MPEG-2, text TV has been digitalized, that is to say, text information has been used as digital information spread in the TS stream. however, in the end, all the digital graphic information is converted into analog VBI information inserted into the normal TV signal to achieve the display of the graphic TV. there are many standards for graphic TV, such as close caption, WSS and so on, European DVB used teletext standard is ITU-R system B teletext specifications. this specification supports multiple European languages and hybrid display of simple images and texts.

Chapter 2 Composition of the DVB System
[This article does not describe the hardware structure of the DVB system in detail, but explains in detail the format and Reuse Principle of the TS code stream according to the description in iso13818. (Enough, Enough)]
Classification of DVB
According to the application of different DVB, mainly divided into DVB-S, DVB-C, DVB-T, DVB-H and other specifications. Different Specifications of the basic principles are the same, different main modulation mode is different:
DVB-S, S is the first character of satelite, used in digital TV satellite broadcast, modulation method is.
DVB-C, C is the first letter of cable, used in urban cable broadcasting. modulation method is all the formats of QAM: 16QAM, 32qam, 64QAM, 128qam, 256qam.
DVB-T, T is the first character of terrestrial, used in terrestrial radio broadcast, modulation method is PSK or 16QAM and 64QAM.
DVB-H, H is the first letter of handheld, used in handheld wireless broadcast, the modulation method is the same as the DVB-T.
In general, different DVB systems only have different front-end systems, and the final digital information is the same, that is, they all adopt the specifications described in iso13818. front-end systems mainly refer to modulation and emission methods. therefore, all the content described here applies to all DVB systems. At the same time, because the ATSC system in the United States is also a small number of changes and a small number of functions on the DVB system, it is also applicable to the ATSC standard.
The basic system description is as follows:
(1) encoding system:
Assume that the signal source has six programs, the DVB system first compresses all the data (video, audio) of the six programs, then, a program called "reuse" is used to reuse programs (PID distribution, that is, assigning different PID numbers to the six programs) to form a package called "PES, then, all the PES packets are encapsulated into the TS bitstream format through the TS stream processing program. Finally, the digital signals in the obtained ts format are modulated to achieve Frequency Multiplexing ), then, the signal is converted into a analog signal through D/A, and then modulated into a high-frequency signal, which is sent by the transmission system.
(2) Decoding System
The receiving system first converts a high-frequency signal to an intermediate-frequency signal, then converts it to a digital signal through a high-speed A/D, and then goes through an anti-modulation program, in this way, the TS code stream is obtained. the ts code stream enters a program called "demultiplexing", which automatically analyzes table information in the TS stream and reads all available PID information, then extract a user-specified PID (the program selected by the user), receive all the digital signals of the program, and ignore other unwanted signals, and then video, the audio signal enters different decompression programs to extract and display the video and audio signals respectively. If the program contains teletext and EPG, it may also process both teletext and EPG, the processing result and video signal are superimposed on the screen.
Packet Concept
(1) The TS stream is based on the packet bit stream format. Each packet is 188 bytes or 204 bytes (generally 188 bytes, the 204-byte format only adds 16 bytes of CRC data to the back of the 188-byte packet. Other formats are the same. The entire TS stream is composed of the following:
Packet 1 packet 2... packet n
In actual use, because the TS stream has a strong internal error processing capability, it generally uses a 188-byte package format, the format of a 204-byte package is generally used in HD programs.
All packet formats are unified, including a packet header and packet datas. the packet header contains the Synchronization Byte (this byte is fixed to 0x47, indicating that the data in this package is correct at the beginning), the unique number of the packet (that is, the PID) and other information. the format is as follows (in C format)
Typedef struct
{
Unsigned sync_byte: 8;
Unsigned transport_error_indicator: 1;
Unsigned payload_unit_start_indicator: 1;
Unsigned transport_priority: 1;
Unsigned PID: 13;
Unsigned transport_scrambling_control: 2;
Unsigned adaptation_field_control: 2;
Unsigned continuity_counter: 4;
} Packet_header;
The preceding structure occupies 32 bits, which is 4 bytes. Therefore, the 4 bytes of the packet header of a TS stream are header information. By analyzing this header information, you can know the attributes of the current packet. the remaining 184 bytes may be video data, audio data, or dvb si information. How can this problem be distinguished? In fact, it is very easy to use the PID information in the header. in the previous chapter, Pat is a program Association table, and its PID is 0x0000. this PID is the PID corresponding to the header here. in other words, if we find that the PID of a packet is equal to 0x0000, it means that this packet is a pat table of DVB instead of video data or audio data.
In fact, when the signal is encoded into a TS code stream, different PIDs are allocated to video and audio data of different programs. for example, if a program has two channels of video and three channels of audio, the distribution of the PID may be Video 1 = 0x100, and video 2 = 0x101, audio 1 = 0x102, audio 2 = 0x103, audio 3 = 0x104, so that the PID in the transmitted ts code stream may have the above PID. therefore, if we need to filter out the first video and second audio in the program, we can do this (pseudocode description ):
Void process_packet (unsigned char * buff)
{
Int pid = getpid (buff );
If (pid = 0x100)
{
Savetovideobuffer (buff + 4 );
}
Else if (pid = 0x103)
{
Savetoaudiobuffer (buff + 4 );
}
Else
{
Printf ("unknown PID! \ N ");
}
}
The problem now is, how do I know the data corresponding to the PID allocated during encoding? This is the analysis and processing of the DVB Si table. Refer to Chapter 3. here we will first look at an example of the actual ts code stream. the data here is obtained by opening the TS code stream file in hexadecimal format using ultraedit. the file is Taiwan-551.ts.
Only the information of three packages is intercepted. Note the red part in the figure. This is the header information of the four bytes of the TS Stream packet. the TS stream uses a 188-byte format for each packet, because the interval between the two header information is 188 bytes (the interval between the first 0x47 and the second 0x47 ). all future packages will be in the 188-byte format, which is a fixed size specified by the DVB ts standard. so what data are contained in these three packages? Let's analyze them by ourselves.
First, check the first package. The header information is "0x47 0x07 0xe5 0x12". As you know, the header information is operated by bit (this is why the TS bit stream can also be called a bit stream). It is important to note that both the definition and transmission are performed by MSB first, that is, the first bit is the highest bit of the data. convert to binary format first:
01000111 00000111 11100101 00010010
Compare the packet_header structure above:
Typedef struct
{
Unsigned sync_byte: 8;
Unsigned transport_error_indicator: 1;
Unsigned payload_unit_start_indicator: 1;
Unsigned transport_priority: 1;
Unsigned PID: 13;
Unsigned transport_scrambling_control: 2;
Unsigned adaptation_field_control: 2;
Unsigned continuity_counter: 4;
} Packet_header;
In comparison, we can find that:
Sync_byte = 01000111, that is, 0x47, which is the Synchronization Byte specified by dvb ts and is fixed to 0x47.
Transport_error_indicator = 0, indicating that the current package has no transmission error.
Payload_unit_start_indicator = 0, meaning please refer to ISO13818-1 standard documentation
Transport_priority = 0, indicating that the current package has a low priority.
PID = 00111 11100101, that is, 0x07e5. What does this mean? I don't know it yet (it's actually a video PID. Refer)
Transport_scrambling_control = 00, indicating that the program is not encrypted
Adaptation_field_control = 01 0x01. For more information, see ISO13818-1.
Continuity_counte = 0010, that is, 0x02, indicates that the number of packages of the same type currently transmitted is 3rd.
Let's take a look at the second package "0x47 0x07 0xe5 0x13". The binary system is 01000111 00000111 11100101 00010011.
Sync_byte = 01000111, that is, 0x47, which is the Synchronization Byte specified by dvb ts and is fixed to 0x47.
Transport_error_indicator = 0, indicating that the current package has no transmission error.
Payload_unit_start_indicator = 0, meaning please refer to ISO13818-1 standard documentation
Transport_priority = 0, indicating that the current package has a low priority.
PID = 00111 11100101, that is, 0x07e5. What does this mean? I don't know it yet (it's actually a video PID. Refer)
Transport_scrambling_control = 00, indicating that the program is not encrypted
Adaptation_field_control = 01 0x01. For more information, see ISO13818-1.
Continuity_counte = 0011, that is, 0x03, indicates that the number of packages of the same type currently transmitted is 4th (note that the PID of the above two packages is 0x07e5, so the continuity_counte here is incremented once)
The third package is "0x47 0x07 0xf1 0x18", and the binary system is 01000111 00000111 11110001.
Sync_byte = 01000111, that is, 0x47, which is the Synchronization Byte specified by dvb ts and is fixed to 0x47.
Transport_error_indicator = 0, indicating that the current package has no transmission error.
Payload_unit_start_indicator = 0, meaning please refer to ISO13818-1 standard documentation
Transport_priority = 0, indicating that the current package has a low priority.
PID = 00111 11100101, that is, 0x07f1. What does this mean? I don't know it yet (it's actually an audio PID. For details, refer)
Transport_scrambling_control = 00, indicating that the program is not encrypted
Adaptation_field_control = 01 0x01. For more information, see ISO13818-1.
Continuity_counte = 1000, that is, 0x08, indicates that 9th packets of the same type are currently transmitted.
See the decoded Program <seekfor MPEG-2 decoder> to read the results of the file:
We can find that the Taiwan-551.ts has a program called "Dimo", its video PID is 0x07e5, audio PID is 0x07e6
There is also a program named "Service 1" without the video PID. Its audio PID is 0x07f1 (indicating a broadcast program rather than a TV program)
This data exactly matches our analysis.
However, I still have some questions. Why does 0x07e5 represent the video PID and 0x07e6 represent one of the audio PID? This is what we mentioned just now. This is what ts streams are allocated during encoding. But how do we know that 0x07e5 represents video rather than audio during decoding? This is the third chapter: Analysis and Processing of DVB Si/psi.

Chapter 3 Analysis and Processing of DVB Si/PSI
Si is the abbreviation of specific information, and PSI is the program specific information. this mechanism allows DVB to send a variety of messages, such as program names, TV station names, various PIDs, private information, and even separately transmit data for data communication. the implementation of these features is attributed to Si/psi.
In the DVB standard, a standard PID is defined to implement Si/psi. these PIDs are retained by the system. Therefore, these PIDs are not used for video PID, audio PID, or other PIDs during DVB encoding. in a simple demultiplexing program, you only need to provide a program for processing Pat and PMT tables to achieve demultiplexing, of course, if you need a more user-friendly interface and more complex functions (such as Ca), you must process other SI tables. for more information about Pat, PMT, SDT tables, and other SI tables, see ISO13818-1 (MPEG-2 system layer standard) and en300468 (DVB Si standard.
The SI retention PID defined by DVB is:
 
The PID in the above table is the PID retained by DVB, and other assigned PID will not occupy these PID. the demultiplexing program only uses tables such as Pat, PMT, and SDT, while Ca applications also need cat and EPG applications need tables such as nit, EIT, TdT, and tot. therefore, when demultiplexing is required, the pseudo code should be written as follows:
Void process_packet (unsigned char * buff)
{
Int pid = getpid (buff );
If (pid = 0x0000)
{
Process_pat (buff + 4 );
}
Else if (pid = ......)
{
}
Else
{
Printf ("unknown PID! ");
}
}
All Tables start with 184 bytes of data in packet, but sometimes a table does not have 184 bytes, in this case, some invalid information may be inserted in the packet to fill the packet so that the entire packet remains 188 bytes. you may also use payload_unit_start_indicator in the header to indicate that the table has an offset position. (when payload_unit_start_indicator = 0, table data starts from the fourth byte in the packet area. Otherwise, there is an offset position, for more information, see the ISO13818-1. The data between 4th bytes and the offset is invalid data filled by the system ).
The following describes the formats of Pat, PMT, and SDT tables for the demultiplexing program.
Pat, program Association Table, program Association Table
The Pat table carries the following information:
(1) TS Stream ID --- transport_stream_id, which indicates the unique stream ID
(2) program channel number-program_number, which indicates a channel in the TS stream. This channel can contain many programs (that is, it can contain multiple video PID and audio PID)
(3) PID of PMT --- program_map_pid, which indicates which PID is used in the current channel as the PID of PMT. Because there can be many channels, the user can define the PID of PMT according to DVB.
The Pat table is defined as follows:
The meanings of each field are as follows:
Table_id: 8 bits, indicating the type of this table, which should be 0x00
Section_syntax_indicator: 1 bit, segment syntax mark, should be '1'
'0': Fixed '0', which is set to prevent the control word conflict with the iso13818video stream format.
Reserved: Reserved 2 bits. The reserved bits are generally '0'
Section_length: the segment size of 12bits. The unit is bytes.
Transport_stream_id: The current stream ID of 16bits, which is unique in DVB. (In fact, many of them are custom ts IDs)
Version_number: 5bits version number, which indicates the version of the current program. this is a very useful parameter. When this field is detected to change, it indicates that the program in the TS stream has changed and the program must be searched again.
Current_next_indicator: 1bit: currently or in the future, the flag is usually '0'
Section_number: 8bits current segment number
Last_section_number: Number of the last segment of 8bits (the functions of section_number and last_section_number are that when the content of Pat is greater than 184 bytes, the Pat table is divided into multiple segments (sections ), the demultiplexing program must perform Pat analysis after receiving all the programs)
Starting from for (), it describes the number of channels in the current stream (N). What is the pmt pid corresponding to each channel. the demultiplexing program needs to receive all channel numbers and corresponding pmt pid in a similar loop, and save the information in the buffer zone. pmt pid must be used in the back processing.
Crc_32: CRC Check value of this section, which is usually ignored. N is a variable, and the calculation method is n = (section_length-9)/4.
From the above analysis, we can find that the Pat table mainly contains the channel number and the pmt pid Number corresponding to each channel. This information will be saved when processing the Pat table, the data will be used later. for example, we can define a Data Structure to save the information:
Typedef struct
{
Int channel_number;
Int pmt_pid;
} Pmt_item;
Pmt_item PMT [64];
PMT, Program Map Table, program shadow table
If a TS Stream contains multiple channels, it will contain multiple PMT tables with different PIDs. The pseudocode for detecting whether PMT is as follows:
Void process_packet (unsigned char * buff)
{
Int I;
Int pid = getpid (buff );
If (pid = 0x0000)
{
Process_pat (buff + 4 );
}
Else if (pid = .....)
{
}
Else
{

For (I = 0; I <64; I ++)
{
If (pid = PMT [I]. pmt_pid)
{
Process_pmt (buff + 4 );
Break;
}
}
}
}
The data contained in the PMT table is as follows:
(1) PID of all video data contained in the current channel
(2) PID of all audio data contained in the current channel
(3) PID of other data associated with the current channel (such as the PID used for digital broadcast and data communication)
PMT is defined as follows:
 
The meanings of each field are as follows:
Table_id: ID of 8bits, which should be 0x02
Section_syntax_indicator: 1bit segment syntax mark, which should be '1'
'0': The fixed value is '0'. If not, the data is incorrect.
Reserved: 2bits Reserved Bit, which should be '00'
Section_length: 16 bits segment length, starting from program_number, to the total number of bytes in crc_32 (inclusive.
Program_number: The Channel Number of 16bits, indicating the channel associated with the current PMT. In other words, the information of the program_number channel is described.
Reserved: 2bits Reserved Bit, which should be '00'
Version_number: version number. If the PMT content is updated, version_number increments by 1 to notify the demultiplexing program to receive program information again. Otherwise, version_number remains unchanged.
Current_next_indicator: The current future identifier, usually 0.
Section_number: current segment number
Last_section_number: Number of the last segment. The meaning is the same as that of the field in Pat. For details, refer to the Pat section.
Reserved: 3bits Reserved Bit, generally '000 '.
Pcr_pid: pcr pid of 13bits. For details, refer to the ISO13818-1. this parameter is not used in the demultiplexing program.
Reserved: Reserved bits of 4bits, generally '20140901'
Program_info_length: Program Information length (N descriptor structures are followed, which can be ignored. This field represents the total length of the descriptor, in bytes)
Then there is the program type and corresponding PID Number contained in the channel.
Stream_type: 8bits stream type, indicating whether it is video, audio, or other data.
Reserved: 3 bits Reserved Bit.
Elementary_pid: Data PID Number corresponding to 13bits (if stream_type is video, This PID is video PID. If stream_type is audio, This PID is audio PID)
Reserved: 4 bits Reserved Bit.
Es_info_length: the length of information similar to program_info_length (followed by N2 descriptive symbols)
Crc_32: The End Of The 32bits segment is the CRC value of this segment, which is generally ignored.
From the above analysis, we can see that as long as we have processed the PMT, we can obtain all the PID information in the channel, such as how many videos and how many audio the current channel contains, and other data, you can also know the PID corresponding to each data.
In this way, if we want to select one of the video and audio to watch, we only need to save the video PID and audio PID of the program to be watched and filter the packet.
The pseudocode that fully implements demultiplexing is as follows:
Int video_pid = 0x07e5, audio_pid = 0x07e6;
Void process_packet (unsigned char * buff)
{
Int I;
Int pid = getpid (buff );
If (pid = 0x0000)
{
Process_pat (buff + 4 );
}
Else if (pid = video_pid)
{
Savetovideobuffer (buff + 4 );
}
Else if (pid = audio_pid)
{
Savetoaudiobuffer (buff + 4 );
}
Else
{
For (I = 0; I <64; I ++)
{
If (pid = PMT [I]. pmt_pid)
{
Process_pmt (buff + 4 );
Break;
}
}
}
}
The above pseudo-code can be used to achieve basic decoding and reuse: detect all channels, detect the PID of all streams, and select a specific program for playing. you only need to read the 188-byte content of each packet and call process_packet () each time to implement simple demultiplexing.
Here, we can summarize the principles of the DVB search platform. (Good! All ears !)
The set-top box first adjusts the high-frequency header to a fixed frequency (such as 498 MHz). If there is a digital signal at this frequency, the coofdm chip (such as mt352) the TS stream data is automatically transmitted to MPEG-2 decoder. the MPEG-2 decoder first synchronizes data, that is, waiting for the arrival of the complete packet. then, the system cyclically finds whether a packet with PID = 0x0000 exists. If yes, it immediately goes to the analysis Pat process and obtains all PMT PIDs. then, it cyclically finds whether PMT exists. If PMT exists, it automatically enters the PMT analysis to obtain and save data of all channels in the frequency band. if Pat is not found or PMT is not found, it indicates that there is no signal in the frequency band and the next frequency scan is performed.
From the above description, we can see that the search frequency of the set-top box is random. To enable the set-top box to search for signals, the TS Stream needs to send Pat and PMT at intervals. in fact, this is what the DVB transmission system does. therefore, the system can immediately search for programs and properly decompress and reuse programs to enable playback at any time. not only that, but other data is also transmitted alternately. for example, the first packet may be Pat, the second packet may be PMT, the third packet may be Video 1, and the fourth packet may be video 2,
As long as the system transmission speed is fast enough (that is, the "bit rate" stuff), real-time playback is no problem.
Although the reuse is realized here, it can be seen that the PID used is a boring number. If the Console requires the user to enter the number by themselves, it is too troublesome and easy to input errors, the operation is not intuitive, even if a menu is made for the user to choose. to solve this problem, the DVB system puts forward an SDT table, which indicates the name of a program and can be associated with the PID in PMT, in this way, you can select a program by directly selecting the program name.
SDT, service description section, service description Section
The SDT provides the following information:
(1) Whether the program is playing
(2) Whether the program is encrypted
(3) name of the program
SDT is defined as follows:
 
The fields are defined as follows:
Table_id: ID of 8bits, which can be 0x42, indicating information about the current stream or 0x46, indicating information about other streams (this parameter is used by EPG)
Section_syntax_indicator: Segment syntax mark, which is generally '1'
Reserved_future_used: 2bits reserved for future use
Reserved: 1 bit Reserved Bit to prevent control word conflicts. Generally, it is '0' or '1'
Section_length: the segment length of 12bits. The unit is bytes. It starts from transport_stream_id and ends with crc_32 (inclusive)
Transport_stream_id: ID of the stream described by 16bits
Reserved: 2bits Reserved Bit
Version_number: Version Number of 5bits. If data is updated, this field is incremented by 1.
Current_next_indicator: indicates the current future. Generally, it is '0', indicating that it is used immediately.
Original_netword_id: original network ID of 16bits
Reserved_future_use: 8bits reserve future BITs
Next is the loop of N program information:
Service_id: 16 bits server ID, which is actually the program_number In the PMT segment.
Reserved_future_used: 6bits reserve future BITs
Eit_schedule_flag: 1bit EIT information. 1 indicates that the current stream implements EIT transmission for this program.
Eit_present_following_flag: the EIT information of 1bits. 1 indicates that the current stream implements EIT transmission of the program.
Running_status: 3 bits running status information: 1-not playing 2-a few minutes later, 3-paused, 4-playing, others-retained
Free_ca_mode: 1bits encrypted information. '1' indicates that the program is encrypted.
Next, the descriptor is usually the service descriptor. After analyzing this descriptor, you can obtain the program name specified by servive_id. for specific formats, see service descriptor in en300468. after analysis, the program name and program number are connected. the set-top box program can use these program names to replace the PID for users to choose, so as to achieve a friendly user interface!