H. 11 format, iOS hard-coded decoding and support for HEVC hard-coded iOS +/.

Source: Internet
Author: User

H. 11 format, iOS hard codec, and support 1,h.264 format for HEVC hard codec in iOS

The network representation layer nal,h.264 flow consists of a frame-by-frame Nalu;
SPS: Sequence parameter set, used for a series of sequential coded images;
PPS: An image parameter set that is used to encode one or more independent images in a video sequence;
These two frames are also independent of the Nalu.

I-frame: keyframes, frames within the frame after encoding , showing a more complete frame;
P-frame: Referring to the previous frame, it is possible to compare the motion estimation of the previous frame with the change part;
B-frame: Will refer to the frame before and after, other similar p-frame. Both the B and P frame contain inter-frame encoding and intra- frame encoding .
Each nalu is delimited by the start Code: (3 bytes) or the XX (4 bytes).

2,ios on the hard coding and decoding of H.

IOS 8 supports hard coding of H. A by Videotoolbox;
The CMSAMPLEBUFFERREF structure is interesting and needs to be understood,


Cmvideoformatdesc contains Sps,pps and other useful information;

    • When encoding, Cvpixelbuffer->cmsamplebufferref (which contains the compressed cmblockbuffer);
    • Decoding, through the vtdecompressionsession to decode, Cmsamplebufferref (contains Cmblockbuffer)->cvimagebufferref (decoded frame data);
2.1 Encoding

Refer to the WWDC 2014 PDF, from the camera or read the cvpixelbuffer->encoder->cmsamplebufferref->nalus of the video file output.

Classic encoder recommended Live push stream Lflivekit, code structure is good, data flow is very clear, is suitable for reading;
Https://github.com/LaiFengiOS/LFLiveKit/blob/master/LFLiveKit/coder/LFHardwareVideoEncoder.m
The coding steps are roughly:

    • 1,vtcompressionsessioncreate via wide, high, Codectype (kcmvideocodectype_h264), coded callback Vtcompressionoutputcallback such as initializing Vtcompressionsessionref instance compressionsession;
    • 2,vtsessionsetproperty set more important encoding properties, such as
      Kvtcompressionpropertykey_maxkeyframeinterval, key frame interval
      Kvtcompressionpropertykey_realtime
      Kvtcompressionpropertykey_averagebitrate
      Kvtcompressionpropertykey_profilelevel
      Kvtcompressionpropertykey_allowframereordering and so on; the last vtcompressionsessionpreparetoencodeframes;
    • 3, add original Cvpixelbuffer and pts to Vtcompressionsessionencodeframe
根据frameCount 计算出的pts
Cmtime Presentationtimestamp = Cmtimemake (Framecount, (int32_t) _configuration.videoframerate);

and whether I-frame:

if 0  = @{(__bridge nsstring *) Kvtencodeframeoptionkey_forcekeyframe: @YES};}
    • 4,vtcompressionoutputcallback callback by parsing cmsamplebufferref processing sps,pps,i-frame and non-i-frame respectively, adding startcode:00 00 00 01 then go through rtmp rollout.
2.2 Decoding

Encoding inverse process, usually in the player and other scenes applications, Nalu + Sps,pps->cmblockbuffer->cmsamplebufferref, and then the cmsamplebufferref wrapped frame data input into Vtdecompressionsessiondecodeframe, Cvimagebufferref directly upload OpenGL ES display via callback).
I wrote a simple iOS H. Decoder stream:
Https://github.com/edisongz/VideoHardwareStreamDecoder

3,ios 11 support for HEVC

IOS One + A9 chip starts support for HEVC, System control Avplayer, and Safari browser support, and HEVC has a 40% higher compression efficiency compared to H.
Hard coding requires IOS11 + A10 chip

Hard decode requires iOS one + A9 chip

IOS 11 New API

Decoderlet hardwaredecodesupported = vtishardwaredecodesupported (KCMVIDEOCODECTYPE_HEVC)//encoderlet error = Vtcompressionsessioncreate (Kcfallocatordefault, 3840, 2160, KCMVIDEOCODECTYPE_HEVC, encoderSpecification, nil, nil, Nil, nil,//using Vtcompressionsessionencodeframewithoutputhandler &session); if error = = Kvtcouldnotfindvideoencodererr {    //no HEVC encoder}

The specific HEVC Videotoolbox hardware codec still needs to be perfected; the subsequent completion is added here:
Https://github.com/edisongz/VideoHardwareStreamDecoder

Summary

The hard codec of the iOS environment is older WWDC's session, this year's WWDC 2017, the addition of hard codec HEVC code, but also need to find a machine under test.

Reference

Hard-coded part of Lflivekit
WWDC Direct Access to Video Encoding and decoding
WWDC working with Heif and HEVC
Github writes its own streaming decoding

H # Format, iOS hard codec, and iOS 11 support for HEVC hard codecs

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.