H. 11 format, iOS hard-coded decoding and support for HEVC hard-coded iOS +/.

H. 11 format, iOS hard codec, and support 1,h.264 format for HEVC hard codec in iOS

The network representation layer nal,h.264 flow consists of a frame-by-frame Nalu;
SPS: Sequence parameter set, used for a series of sequential coded images;
PPS: An image parameter set that is used to encode one or more independent images in a video sequence;
These two frames are also independent of the Nalu.

I-frame: keyframes, frames within the frame after encoding , showing a more complete frame;
P-frame: Referring to the previous frame, it is possible to compare the motion estimation of the previous frame with the change part;
B-frame: Will refer to the frame before and after, other similar p-frame. Both the B and P frame contain inter-frame encoding and intra- frame encoding .
Each nalu is delimited by the start Code: (3 bytes) or the XX (4 bytes).

2,ios on the hard coding and decoding of H.

IOS 8 supports hard coding of H. A by Videotoolbox;
The CMSAMPLEBUFFERREF structure is interesting and needs to be understood,

Cmvideoformatdesc contains Sps,pps and other useful information;

• When encoding, Cvpixelbuffer->cmsamplebufferref (which contains the compressed cmblockbuffer);
• Decoding, through the vtdecompressionsession to decode, Cmsamplebufferref (contains Cmblockbuffer)->cvimagebufferref (decoded frame data);

2.1 Encoding

Refer to the WWDC 2014 PDF, from the camera or read the cvpixelbuffer->encoder->cmsamplebufferref->nalus of the video file output.

Classic encoder recommended Live push stream Lflivekit, code structure is good, data flow is very clear, is suitable for reading;
Https://github.com/LaiFengiOS/LFLiveKit/blob/master/LFLiveKit/coder/LFHardwareVideoEncoder.m
The coding steps are roughly:

• 1,vtcompressionsessioncreate via wide, high, Codectype (kcmvideocodectype_h264), coded callback Vtcompressionoutputcallback such as initializing Vtcompressionsessionref instance compressionsession;
• 2,vtsessionsetproperty set more important encoding properties, such as
  Kvtcompressionpropertykey_maxkeyframeinterval, key frame interval
  Kvtcompressionpropertykey_realtime
  Kvtcompressionpropertykey_averagebitrate
  Kvtcompressionpropertykey_profilelevel
  Kvtcompressionpropertykey_allowframereordering and so on; the last vtcompressionsessionpreparetoencodeframes;
• 3, add original Cvpixelbuffer and pts to Vtcompressionsessionencodeframe

根据frameCount 计算出的pts

Cmtime Presentationtimestamp = Cmtimemake (Framecount, (int32_t) _configuration.videoframerate);

and whether I-frame:

if 0  = @{(__bridge nsstring *) Kvtencodeframeoptionkey_forcekeyframe: @YES};}

• 4,vtcompressionoutputcallback callback by parsing cmsamplebufferref processing sps,pps,i-frame and non-i-frame respectively, adding startcode:00 00 00 01 then go through rtmp rollout.

2.2 Decoding

Encoding inverse process, usually in the player and other scenes applications, Nalu + Sps,pps->cmblockbuffer->cmsamplebufferref, and then the cmsamplebufferref wrapped frame data input into Vtdecompressionsessiondecodeframe, Cvimagebufferref directly upload OpenGL ES display via callback).
I wrote a simple iOS H. Decoder stream:
Https://github.com/edisongz/VideoHardwareStreamDecoder

3,ios 11 support for HEVC

IOS One + A9 chip starts support for HEVC, System control Avplayer, and Safari browser support, and HEVC has a 40% higher compression efficiency compared to H.
Hard coding requires IOS11 + A10 chip

Hard decode requires iOS one + A9 chip

IOS 11 New API

Decoderlet hardwaredecodesupported = vtishardwaredecodesupported (KCMVIDEOCODECTYPE_HEVC)//encoderlet error = Vtcompressionsessioncreate (Kcfallocatordefault, 3840, 2160, KCMVIDEOCODECTYPE_HEVC, encoderSpecification, nil, nil, Nil, nil,//using Vtcompressionsessionencodeframewithoutputhandler &session); if error = = Kvtcouldnotfindvideoencodererr {    //no HEVC encoder}

The specific HEVC Videotoolbox hardware codec still needs to be perfected; the subsequent completion is added here:
Https://github.com/edisongz/VideoHardwareStreamDecoder

Summary

The hard codec of the iOS environment is older WWDC's session, this year's WWDC 2017, the addition of hard codec HEVC code, but also need to find a machine under test.

Reference

Hard-coded part of Lflivekit
WWDC Direct Access to Video Encoding and decoding
WWDC working with Heif and HEVC
Github writes its own streaming decoding

H # Format, iOS hard codec, and iOS 11 support for HEVC hard codecs