Format of RTP payload applied to bundled MPEG

This document describes a suitable for bundling, MPEG-2 encoding, RTP protocol can be applied to the video and audio frequency
The payload type of the data. This is the second version. For this type of payload, when it is used in a VoD application system,
Bundling has obvious advantages. This advantage is important enough to sacrifice the modularization of the separated audio video streams.
This type of payload may be used.
1. Introduction
This document describes an audio and
The bundled packaging scheme of video streams.

MPEG-2 international standards consist of three levels: audio, video and system [2]. The audio and video layers define the corresponding
Syntax and semantics of "elementarystreams. The system layer supports synchronization and crossover of multiple compression streams to reduce latency.
The initialization and Management of the dashboard, as well as time identification. Rfc2250 [3] describes how to transmit basic audio and video streams separately,
That is, the packaging technology used for transmitting a stream. The stream is defined at the system layer and RTP is used.
Bundling and packaging solutions are required, because for some important applications, it has several advantages over other solutions. These
Applications include VOD, where audio and video are always used together. Compared with audio and video packaging,
Its advantages include:

1. Each "program" (such as bundled audio/video) uses a unique port. This method adds service-available
The number of streams, for example, from a VOD Server. In addition, it eliminates the need to apply two ports on the client to separate audio and video
Performance collision during frequency stream.

2. Provides the explicit synchronization of audio and video (implicitsynchronization ). When audio and video data
Cross-format storage on the server is obviously convenient.

3. The overhead is reduced ). Since the use of large packages will increase the impact of loss and delay
There is a small increase in the total overhead for audio packages. The A/V bundle format can be reduced by about 1% in total. Considering MPEG-2
The encoding material uses a high rate. For example, when it is 4 Mbps, the number of digits saved is 40 kbps, which can provide audible sound.
Improve the frequency or video quality.

4. The buffer size of the receiver can be reduced comprehensively. Audio and video streams may experience different latencies during transmission.
The receiver's buffer must be designed to fit the maximum value of these latencies. For example, let's assume that two buffers are used,
The size of each stream is B, and the probability P for each stream to be transmitted separately is sufficient. Buffer Capacity of the same size
The probability of receiving two streams is that P is multiplied to receive B of the first stream and can receive B of the second stream.
Conditional probability. The probability of this condition is generally smaller than that of reaching the same probability level with a large buffer zone.

5. It can help to control the overall bandwidth used by an A/V program.

In addition, compared with the packaging of transmission flow, its advantages are:

1. Reduce the total overhead. It does not contain information on the system layer, which is redundant for RTP. (Essentiallythey
Addresssimilarissues)

2. Easier error recovery. Because structured packaging and application-layer frame separation (applicationlayerframing)
(ALF) The rules are consistent, and the loss masking (lossconcealment) and error recovery (errorrecovery) are simpler and
Valid.
2. encapsulation of bundled MPEG videos and audios
Video encapsulation follows the same encapsulation as described in [3] For mpegelementarystreams.
. Specifically:

1. When mpegvideo_sequence_header appears, it will always start with a RTP payload.

2. When an mpeggop_header appears, it will always start at the beginning of a RTP payload, or
Followed by a video_sequence_header.

3. When an mpegpicture_header appears, it will always be at the beginning of a RTP payload, or
Followed by a gop_header.

In addition, you also need:

4. Each package must also contain an integer of video clips (videoslices ).

The application has the responsibility to adjust the size and quantity of fragments placed in each RTP package, so that the underlying layer is not generated.
Segment (lowerlevelfragmentation ). When the packetizer of the transmitter (transmitrer) has a certain complexity
This approach simplifies the receiver when the degrees increase ). Considering that a piece may be smaller than a microblock
(Macroblock) is the same, which can prevent segmentation in most cases. If the size of a package exceeds the maximum transfer order of the path
(Pathmaximumtransmissionunit (path-MTU) [4], although the type of the payload depends on
Low protocol layer, but this may lead to problems with packetclassification of integrated services (such as RSVP
).

The video data follows a sufficient number of complete audio frames to cover the time interval of the video segment in the package. Example
For example, if the first package contains three video clips of 1/900 seconds and uses layeri audio at a 44.1khz Sampling Rate
Encoding, the audio that is sometimes 384/44100 seconds long is contained in this package. Since the length of the audio frame (8.71
Msec.) is longer than the length (3.33msec.) of the video segment contained in this package, you can skip this package in the next few packages.
Includes any number of audio frames until the duration of the video segment in a package extends beyond the previously transmitted audio frame. In
We recommend that you resend the nearest audio frame in the "audio-free" package to restore the packet loss (resilence ).
In addition, the application has the responsibility to adjust the size of the bundle according to the minimum MTU size to avoid segmentation.
2.1.rtp is applied to the solid header encapsulated by BMP.
The following RTP Header fields must be used:

Payloadtype: A unique payloadtype number, which may be dynamic and
It should be assigned to BMP.

M-bit (Mbit): set to include the package at the end of the image.

Timestamp: The 32-bit 90 kHz timestamp indicates the sampling time of the MPEG image. If graph B appears,
It may not increase monotonically. For packages that contain the same image, it is the same. For,
The package of the extension and/or GOP header. The timestamp is the timestamp of the subsequent image.
2.2.bmp header details:
0123
01234567890123456789012345678901
+- +-+
Pnmbzaudiolengthaudiooffset
+- +-+
Mbz

P: Image Type picturetype (2 bits). I (0), P (1), B (2 ).

N: changes the header data (1bit ). If any part of the video sequence, extension, GOP, and header data is different from the prepaid data
The sent header, Which is set. This bit is reset when all header data is resold (see appendix 1 ).

Mbz: The value must be 0. Reserved for future use.

Audiolength audio length:
(10 bits) represents the length of the audio data in the package in bytes. The audio data can start from
The total length of the received packet minus "audiolength" is obtained.

Audiooffset audio offset:
(16 bits) the offset between the starting position of the audio frame and the RTP timestamp of the packet based on the number of audio samples (
Multi-channelsources. For this purpose, a group of samples covering all channels are
Sample ).

The audio offset is a signed integer in two complementary forms. When the 44.1khz audio is sampled, it allows
One ~ +/-750msec offset. When the video frame rate is very low (for example, 1 frame per second), this offset
The format of the payload may be insufficient.

If frame B appears, the audio frames are not re-ordered together with the video ). Instead
The input sequence is packaged together with the video frame (for example, the audio segment packaged together with a video segment corresponding to the P image may belong
B diagram that will be later transmitted and presented at the same time as the audio segment ). Although video segments are reordered
The audio offset of a specific audio segment is still relative to the RTP timestamp in the packet containing the audio segment.

Since a dedicated image counter, like the "time reference" field in [3], is not included in this payload format,
The missing GOP header may not be detected. The only impact of this may be that some edited video clips keep up
The B diagram after the lost GOP header is incorrectly decoded.
3. Security considerations
The use of RTP packets in the payload format defined in this document is subject to the security considerations discussed in RTP specification [1.
This means that the confidentiality of a media stream can be achieved through encryption. Because the data compression used in this payload format applies
End-to-End (end-to-end) encryption can be performed after compression, so that there is no conflict between the two operations.

This payload type does not show any significant inconsistency in the computing complexity of the receiver packet processing.
(Non-uniformity) does not cause potential denial-of-service threats

Looking back at the security of this payload format, no additional considerations were found beyond the RTP specification.

Appendix 1. Error Recovery)
Packet loss can be combined from the serial number (sequencenumber) and timestamp fields in the RTP fixed header.
Detected. The loss range can be determined by the packet timestamp, slicenumber, and the level of the first segment.
Location (horizontallocation ). The part number and horizontal position can be determined by the part header and the first microblock.
(Macroblock) increments, all of which are at fixed bitpositions ).

If all the parts that constitute the lost data are from the same image, the new data that follows the Lost part can
It is simply sent to the video decoder, which usually repeats the missing pixels in the previous image. The next audio frame must be included in
The time stamp and audio offset in the receiving package determine the proper time for playing. Suitable audio frames (for example, representing background noise)
Audio) may need to be fed back to the location where the audio frame is lost in the audio decoder to maintain the lip-synch, and/or mask.
The impact caused by the loss of the CAP.

If the new data after the loss comes from the next image (for example, the entire image is not lost) and N bits are not set
The previously Received Header for a specific image type (determined by the P bit) can be sent to the video decoder, followed by the new
. If n-bit is set, the header is available for the receiver unless it is made available through another channel.
To use the data to delete until the start code of a new image.

If the data of more than one image is lost and the header is unavailable, you can use resynchronization.
(Resynchronization) to a new video sequence header, unless n is 0 and for each insert of the same type
An image (interveningpicture) accepts at least one packet, and each n-bit in these images is 0.

If the correct header is available for the lost image in case of any serious packet loss, they can be sent
To the video decoder, and can use the received data regardless of the N-bit value or the number of lost images.

Appendix 2. Resynchronization)
As described in [3], frequent use of video sequence headers makes it possible to add any number of geographic parameters to programs. It also shrinks
A short synchronization time after serious loss.

Reference
[1] schulzrinne, H., casner, S., Frederick, R., andv. Jacob,
"RTP: atransportprotocolforreal-timeapplications ",
Rfc1889, january1996.

[2] ISO/iecinternationalstandard13818; "genericcodingof
Movingpicturesandassociatedaudioinformation ",
November1994.

[3] Hoffman, D., Fernando, G., Goyal, V., andm. civanlar, "RTP
Payloadformatformpeg1/mpeg2video ", rfc2250,
January1998.

[4] mogul, J., ands. Deering, "pathmtudiscovery", rfc1191,
November1990.