Turn HEVC reference frame management (RPS)

Analyze the management of the HM Reference frame:

HEVC uses the reference frame set (RPS) technology to manage decoded frames as a reference for subsequent images. Unlike the previous video encoding standard, the RPS technology in HEVC transmits the state change of each frame in the DPB through the stream of titles that begin directly at each frame, while the sliding windows and Mmco in the H.264/AVC (Memory Managementcontrol Operation) These two reference frame management techniques are implemented by transmitting the relative changes of the DPB of each piece, which can have a lasting impact if data loss occurs.

1. Basic knowledge of reference frame management

H.263, H.264/AVC and HEVC adopt multi-frame reference technology to improve the accuracy of inter-frame prediction. Decoding the completed image is cached in the DPB for subsequent frames to be used as reference images, so the standard needs to be managed for the images in the DPB. Each image in DPB has three states "Unused for reference", "used for short-termreference" and "used for long-term reference", which convert between three states, is controlled by decoding the image tagging process. H. There are two ways to use sliding windows and Mmco in 264/AVC. Sliding window management, as the name implies, is to DPB can be stored in the window, with the current decoding of the image, in first-in, first-out way, the new decoded image will be moved out of the decoded image of the window, so that the DPB is the most recent decoding of multiple images. Mmco is the status tag of an image in a DPB by transmitting a control command in a stream, which can mark a "used for reference" (short-term or long-term) as "Unused for Reference can also mark the current frame or "used for short-termreference" frame as "used for long-term reference" and so on.

2. HEVC Reference frame Set technology (RPS)

In order to enhance the anti-error ability of reference frame management, HEVC adopts reference detection technology, which transmits the state change of each frame in the DPB in the first stream of each frame, and describes the reference frame that the current frame and the subsequent frame may use in DPB, and describes the identification of the POC as a frame. Therefore, it does not need to rely on the DPB state of the front frame, thus enhancing the performance of anti-error. In short, the first slice of each frame contains an RPS, which consists of a set of "Δpoc" (the difference between the current frame POC and its POC) and "whether it is used as a reference for the current frame/subsequent reconnaissance", which DPB the reference frame based on the description of RPS in the title. For example, Poc=4 's B-frame, can only refer to the poc=0 I-frame, its RPS is {(Δpoc =3,usedbycurpic)}; Poc=3 B-Frame, can refer to the p0c=4 and p0c=2 two B-frame, its RPS {(Δpoc = -1,usedbycurpic), (Δpoc=1,usedbycurpic)}; If the poc=0 I frame no longer appears in RPS, it is labeled "U Nused for reference "and then removed from DPB.

Reference Frame RPS Update:

A) before decoding the p0c=1 frame, there are three reference frames in the DPB, respectively, poc=0,4,2 frames;

b) When the RPS information transmitted in the P0c=1 's title is received (describing the decoded p0c=1 frame, the state of the DPB, that is, the frame of poc=0 and 2 is used as the short-term reference for the current frame, and the p0c=4 frame is used as a reference for subsequent frames), the status of the frame in the DPB is marked, Mark the frames of poc=0 and 2 as "used for short-termreference", and retain the poc=0 frame for subsequent reference, then decode the p0c=1 frame, decode it, and put the p0c=1 frame in DPB. At this time, there are poc=0, 4, 2, 14 frames in the DPB;

c) When the RPS information transmitted in the P0c=3 's title is received (describing the frame decoding p0c=3, DPB, p0c=2 and 4 are used as the short-term reference for the current frame, the poc=0 frame is used as a reference for subsequent frames, and the poc=1 frame is no longer used as a reference). Mark the frame in DPB, mark the frames of p0c=2 and 4 as "used for short-termreference move the p0c=1 frame out of DPB, then decode the p0c=3 frame, decode it, and put it in DPB." At this point, there are poc=0,4, 2, 34 frames in the DPB.

3. RPS forecasts

As can be seen from the above description, when using the POC value to describe the reference frame, the difference between multiple ΔPOC values in RPS of different frames is the same, for example, in the RPS description of the frame of the POC 1, the ΔPOC value is in the RPS description of {1, -3,-1}, and the POC 3 frame, respectively. The difference between the Δpoc value {3,-1,1} and Δpoc is 2. Therefore, in the RPS description of the different frames, the ΔPOC value can still be predicted, that is, Deltarps =δpoc (Rpscurr)-δpoc (RPSREF) to further reduce the transmission cost.

4. Multi-reference frame configuration in HM

The reference software HM uses the HIERACHICAL-B GOP structure in the random access configuration, the reference frame number is 4 (or 3), 3-3, but in the encoding configuration, the active reference frame number is 2, that is, only the last two reference frames for motion estimation and motion compensation. Similarly, in the Lowdelay configuration, the number of reference frames is 4, and the number of reference frames for the activity is 4.

5, HM Multi-Reference frame optimization selection

In the previous analysis, HEVC and its reference software HM both support multi-reference frame technology, HM can be configured with 4 or 2 active reference frames, which makes each PU motion estimation complexity multiplied. Traditional block matching techniques are still used in the HEVC reference software to search motion vectors and select the best prediction direction and reference frame index. The process can be summarized briefly as follows:

(1) starting from listx=0 (i.e. forward prediction), the reference index Ireflndex is set to 0;

(2) Under the current Ireflndex, generate advanced motion vector prediction (amvp:advanced motionvector prediction) candidate list, and make the selection of AMVP index;

(3) with the AMVP selected in (2) as the starting point, the motion estimation is carried out at the cost of SA (T) d+λpred*bits at the current Ireflndex frame, and the best motion vectors and minimum cost are obtained.

(4) Set Irefindex=irefindex+l, repeat steps (2) and (3), get the best motion vector and the minimum cost under the current ireflndex; until all the references in the frame queue listx are labeled "Used for Reference" The reference frames are calculated with the best motion vectors and minimum cost, and the lowest cost Ireflndex are selected as the best reference index under the current forecast direction.

(5) Listx=1 (i.e., forward prediction), repeat (2) to (4) step, and select the least expensive Ireflndex as the best reference index under the current forecast direction;

(6) Fix the best reference index and motion vectors in List0 (or List1) as one of two-way predictions, perform (2) to (4) steps for all reference indexes in List1 (or list0), and find the best bidirectional Predictive Reference index and motion vectors.

(7) Comparing the minimum cost of forward prediction, back prediction and bidirectional prediction, the best prediction direction is chosen.

Turn HEVC reference frame management (RPS)