Let WEBRTC support H264 codec

Recently experiment how to let WEBRTC support H264 code, record, for people who need reference.

To illustrate, I was compiling the WebRTC under Ubuntu Server 14.04, using the native (c + +) API to develop WebRTC applications. So my adjustments are based on the native code.

The end result is that the browser can send a video with H264 or receive H264 video.

Note that WebRTC uses OpenH264 to do encoder (see h264_encoder_impl.cc), using FFmpeg to do decoder (see h264_decoder_impl.cc). Code version

This article corresponds to the code is February 8, 2017, you can use gclient revinfo-a to view the specific version, as follows:

compilation option Adjustment

WEBRTC can support H264, but it is not turned on by default when compiling under Linux.

rtc_use_h264, this switch controls whether to use H264 (the macro webrtc_use_h264 in the corresponding C + + code), defined in the Webrtc/webrtc.gni file:

rtc_use_h264 = proprietary_codecs &&!is_android &&!is_ios

Proprietary_codecs is defined in Build/config/features.gni:

Proprietary_codecs = is_chrome_branded | | Is_chromecast

I compile under Linux, branded default is Chromium, so, proprietary_codecs default is False.

Want to go, had to pass GN Gen when passed into the args to adjust the more convenient, use the following command to generate the Ninja build file:

GN Gen Out/h264debug--args= "Proprietary_codecs=true"

Once executed, you can use the following command to verify:

GN args Out/h264debug--list=proprietary_codecs
gn args Out/h264debug--list=rtc_use_h264

Seeing that current Value is true indicates that this option is already in effect.

Open rtc_use_h264, OpenH264 encoding support will be enabled.

WEBRTC internal will use FFmpeg to decode H264 (see h264_decoder_impl.cc), ffmpeg associated with an option--rtc_initialize_ffmpeg, this also must be true, otherwise ffmpeg av The codec will not initialize and cannot be used.

The RTC_INITIALIZE_FFMPEG definition is defined in Webrtc/webrtc.gni:

Rtc_initialize_ffmpeg =!build_with_crhome

Because we compile for native development, build_with_chrome defaults to False, so rtc_initialize_ffmpeg defaults to true without adjustment.

The rtc_initialize_ffmpeg switch corresponds to a macro webrtc_initialize_ffmpeg in a C + + code.

To use FFmpeg's H264 decoder feature, you also need to modify a macro: Ffmpeg_h264_decoder. In the Config.h file, the path is third_party/chromium/config/chromium/linux/x64. The original definition is as follows:

#define Config_h264_decoder 0

Change to 1. This way the Avcodec_register_all () method registers the H264 decoder with the system.

Wait, there is actually a part of the very important work to be done. Because Linux compiles WEBRTC, the default generated ninja build file, there is no ffmpeg h264 decoder corresponding source code, so even if you open the Ffmpeg_h264_decoder does not work, you have to modify third_party/f Fmpeg/ffmpeg_generated.gni file, find the conditions that contain the H264 and open it.

Note: Because I did not open H264 support at the beginning of the compilation, after the Ffmpeg_generated.gni file was modified, a new directory was specified with the GN Gen Generation Ninja build file, and then the FFmpeg related Ninja file (three) was copied to The original build directory, and then use the Ninja ffmpeg command to compile the so file. order adjustment of codec

When the Web page uses WebRTC to send the SDP, the default codec order is: VP8 VP9 H264

In C + + code, the first one to match is selected by default (from Peerconnection::createanswer/setremotedescription two methods, you can see). So, we're going to modify the C + + code to change this selection logic.

WebRtcVideoChannel2 (webrtcvideoengine2.cc) used by codec, from The Internalencoderfactory Class (internalencoderfactory.cc), either as the sender or receiver, is encoded in this form.

In the Internalencoderfactory constructor, you can adjust the order of the codec, the default code is as follows:

Supported_codecs_.push_back (Cricket::videocodec (Kvp8codecname));
if (webrtc::vp9encoder::issupported ())
  Supported_codecs_.push_back (Cricket::videocodec (KVp9CodecName));
if (webrtc::h264encoder::issupported ()) {
  Cricket::videocodec codec (kh264codecname);
  TODO (Magjed): Move setting these parameters into Webrtc::h264encoder
  //instead.
  Codec. SetParam (Kh264fmtpprofilelevelid,
                 kh264profilelevelconstrainedbaseline);
  Codec. SetParam (kh264fmtplevelasymmetryallowed, "1");
  Supported_codecs_.push_back (Std::move (codec));
}

Supported_codecs_.push_back (Cricket::videocodec (Kredcodecname));
Supported_codecs_.push_back (Cricket::videocodec (Kulpfeccodecname));
....

Just adjust the H264 codec to the front.

Made this adjustment, Native app as one end of the sending video, in the SDP negotiation, H264 support will be placed in front, the other end if support H264 decoding, will prefer H264 format, on both sides can be H264 to interactive video stream.

When the browser acts as one end of the sending video, the order in which it sends the video format is VP8, VP9, h264,native C + + code that adjusts the order of the local codec in this order, and the code is in mediasession.cc:

Template <class c> static void Negotiatecodecs (const std::vector<c>& Local_codecs, Const std::vector<c>& Offered_codecs, std::vector<c>* negotiated_codecs)
    {for (const c& ours:local_codecs) {C theirs; Note that we intentionally only find one matching codec for each of our//local codecs, in case the remote offer C
    Ontains duplicate codecs.
      if (Findmatchingcodec (Local_codecs, Offered_codecs, Ours, &theirs)) {C negotiated = ours; Negotiated.
      Intersectfeedbackparams (theirs); if (Isrtxcodec (negotiated)) {Const Auto Apt_it = Theirs.params.find (Kcodecparamassociatedpayloadtype)
        ;
        Findmatchingcodec shouldn ' t return something with no apt value.
        Rtc_dcheck (Apt_it! = Theirs.params.end ()); Negotiated.
      SetParam (Kcodecparamassociatedpayloadtype, Apt_it->second); } if (Codecnameseq (Ours.name.c_str (), KH264CodecName) {webrtc::h264::generateprofilelevelidforanswer (Ours.params, Theirs.params, &negot
      Iated.params);
      } negotiated.id = Theirs.id;
      Negotiated.name = Theirs.name;
    Negotiated_codecs->push_back (Std::move (negotiated)); }}//Rfc3264:although the Answerer list the formats in their desired//order of preference, it is recommende  D that unless there are a//specific reason, the Answerer list formats in the same relative order//They were present
  In the offer.
  Std::unordered_map<int, int> payload_type_preferences;
  int preference = static_cast<int> (Offered_codecs.size () + 1);
  for (const c& codec:offered_codecs) {payload_type_preferences[codec.id] = preference--; } std::sort (Negotiated_codecs->begin (), Negotiated_codecs->end (), [&payload_type_preferences] (cons
               T c& A, const c& b) {return payload_type_preferences[a.id] >      Payload_type_preferences[b.id];
});
 }

The final sort call, which adjusts the order of the decoding formats we support, is based on the codec order of the sending side. So, we also need to modify this here, to remove the sorted parts, or to H264 to remove. re-compiling

You can compile a specific module by using the following command:

Ninja PC (for mediasession.cc)
Ninja Media (for internalencoderfactory.cc and webrtcvideoengine2.cc)
Ninja FFmpeg (Pin to FFmpeg)

Then compile your own native app.

Related reading: WebRTC study material Daquan Ubuntu 14.04 under the compilation WebRTC WebRTC source Turnserver use method open WebRTC log (native API)