Let WebRTC use external audio and video codecs

WebRTC supports the use of its own codec (limited to native development), audio, video can be. Here the video coding as an example to analyze the corresponding source code in the WebRTC.

Createpeerconnectionfactory

In Webrtc/api/peerconnectioninterface.h there is a method Createpeerconnectionfactory, the prototype is as follows:

inline rtc::scoped_refptr<PeerConnectionFactoryInterface> CreatePeerConnectionFactory(    rtc::Thread* worker_and_network_thread,    rtc::Thread* signaling_thread,    AudioDeviceModule* default_adm,    rtc::scoped_refptr<AudioEncoderFactory> audio_encoder_factory,    rtc::scoped_refptr<AudioDecoderFactory> audio_decoder_factory,    cricket::WebRtcVideoEncoderFactory* video_encoder_factory,    cricket::WebRtcVideoDecoderFactory* video_decoder_factory);

As you can see, the last four parameters of Createpeerconnectionfactory allow us to provide our own codec factory. This allows us to implement our own factory, create our own encoder (decoder) in factory, and use our WebRTC (encoder) inside decoder.

The code snippet resembles the following:

    rtc::scoped_refptr<webrtc::PeerConnectionFactoryInterface> factory = webrtc::CreatePeerConnectionFactory(            current_thread, current_thread,             your_adm,             your_audio_encoder_factory,             your_audio_decoder_factory,            your_video_encoder_factory,            your_video_decoder_factory);    rtc::scoped_refptr<PeerConnectionInterface> peer_connection = factory->CreatePeerConnection(            config, constraints, NULL, NULL, your_observer);

Webrtcvideoencoderfactory

This interface is defined in Webrtcvideoencoderfactory.h (webrtc/media/engine):

Class Webrtcvideoencoderfactory {public://This VIDEOCODEC class is deprecated. Use Cricket::videocodec directly//instead and the corresponding factory function.  See//http://crbug/webrtc/6402 for more info.    struct VIDEOCODEC {webrtc::videocodectype type;    std::string name; Videocodec (Webrtc::videocodectype t, const std::string& NM): Type (t), name (nm) {} videocodec (WEBRTC::VIDEOC        Odectype T, const std::string& NM, int w, int h, int fr)  : Type (t), name (NM) {}};  Virtual ~webrtcvideoencoderfactory () {}//TODO (Magjed): Make these functions pure virtual when every external client// Implements it.  See http://crbug/webrtc/6402 for more info.  Caller takes the ownership of the returned object and it should be released//by calling Destroyvideoencoder ().  Virtual webrtc::videoencoder* Createvideoencoder (const cricket::videocodec& codec); Returns a list of supported codecs In order of preference.  Virtual const std::vector<cricket::videocodec>& SUPPORTED_CODECS () const;  Caller takes the ownership of the returned object and it should be released//by calling Destroyvideoencoder (). Deprecated:use Cricket::videocodec as argument instead.  See//http://crbug/webrtc/6402 for more info.  Virtual webrtc::videoencoder* createvideoencoder (webrtc::videocodectype type);  Returns a list of supported codecs in order of preference. Deprecated:return Cricket::videocodecs instead.  See//http://crbug/webrtc/6402 for more info.  Virtual const std::vector<videocodec>& codecs () const; Returns true if Encoders created by this factory of the given codec type//would use internal camera sources, meaning That's they don ' t require/expect//frames to be delivered via Webrtc::videoencoder::encode.  This flag was used//as the Internal_source parameter to//WEBRTC::VIEEXTERNALCODEC::REGISTEREXTERNALSENDCODEC. virtual BOOL EncodertyPehasinternalsource (webrtc::videocodectype type) const {return false; } virtual void Destroyvideoencoder (webrtc::videoencoder* encoder) = 0; Private://TODO (Magjed): Remove these. They is necessary in order to return a const//reference to a std::vector in the default implementations of codecs () an d//Supported_codecs ().  See http://crbug/webrtc/6402 for more info.  mutable std::vector<videocodec> Encoder_codecs_; mutable std::vector<cricket::videocodec> Codecs_;};

Implement Webrtcvideoencoderfactory interface, pass to Createpeerconnectionfactory can. You can refer to Internalencoderfactory.h and internalencoderfactory.cpp under the Webrtc/media/engine directory.

External codecs and internal relationships

In the case of video coding, there is a segment code in webrtc/media/engine/webrtcvideoengine2.cc that illustrates the relationship between an external video encoder and an internal video encoder:

Webrtcvideochannel2::webrtcvideosendstream::allocatedencoderwebrtcvideochannel2::webrtcvideosendstream::  Createvideoencoder (const videocodec& codec) {rtc_dcheck_run_on (&thread_checker_);  Do not re-create encoders of the same type. if (codec = = Allocated_encoder_.codec && allocated_encoder_.encoder! = nullptr) {return Allocated_encoder  _;  }//Try creating external encoder. if (external_encoder_factory_! = nullptr && Findmatchingcodec (external_encoder_factory_->supported_    Codecs (), codec)) {webrtc::videoencoder* encoder = External_encoder_factory_->createvideoencoder (codec);  if (encoder! = nullptr) return Allocatedencoder (Encoder, codec, TRUE/* is_external */);  }//Try creating internal encoder. if (Findmatchingcodec (Internal_encoder_factory_->supported_codecs (), codec)) {if (parameters_.encoder_ Config.content_type = = Webrtc::videoencoderconfig::contenttype::kscreen && ParametErs_.conference_mode && Usesimulcastscreenshare ()) {//TODO (sprang): Remove This adapter once LIBVPX support      s simulcast with//Same-resolution substreams.      Webrtcsimulcastencoderfactory adapter_factory (Internal_encoder_factory_.get ()); Return Allocatedencoder (adapter_factory.    Createvideoencoder (codec), codec, false/* is_external */); } return Allocatedencoder (Internal_encoder_factory_->createvideoencoder (codec), codec, false/* IS_EX  ternal */);  }//This shouldn ' t happen, we should is not being trying to create something we don ' t//support.  Rtc_notreached (); Return Allocatedencoder (NULL, Cricket::videocodec (), false);}

As you can see, this will first attempt to call External_encoder_factory_ to create an external encoder, if created successfully, with external, if failed, then look inside.

The external_encoder_factory_ here is the Encoder factory instance that we passed in when we called Createpeerconnectionfactory.

Related reading:

• WEBRTC Study Material Daquan
• Ubuntu 14.04 under compilation WebRTC
• How to use Turnserver in WEBRTC source code
• Open the WebRTC log (native API)
• Let WEBRTC support H264 codec
• GN and Ninja of the WEBRTC compiler system
• WEBRTC the GN files of the compiling system
• How to enable trace_event in WebRTC codes

Let WebRTC use external audio and video codecs