Local Video collection of WEBRTC

Reprint Please specify source: http://www.cnblogs.com/fangkm/p/4374610.html

The first two articles describe the running process of WEBRTC and the use of the framework interface, and then begin to analyze the local audio and video collection process. Due to the large space, video capture and audio capture are divided into two blog posts, where the video capture process is analyzed first. Analysis of the time of the first analysis of the WEBRTC of the original video capture process, and then incidentally mention chromium to WEBRTC video collection adaptation, so as to better understand the WEBRTC interface design.

1. WEBRTC native video Capture

Before introducing the acquisition of video equipment, the first thing to do is to analyze the devicemanager structure of WEBRTC, because the abstraction interface Videocapturer WEBRTC native implementation of video acquisition is created by it. The functionality of this class also includes information about enumerating audio and video devices. The structure is as follows:

Limited to space, the UML does not mark all the functions of the Devicemanagerinterface interface interface, including: To obtain a list of audio input/output devices, to obtain a list of video input devices, based on device information to create Videocapturer video capture objects. Due to the acquisition of a list of hardware devices involving platform-related calls, the implementation under the Windows platform is the Win32devicemanager class (a static method that can call Devicemanagerfactory create () Returns the corresponding DeviceManager object for the current platform). Focus on Devicewatcher, as the name implies, its function is to monitor the change of equipment. Implementation under the Windows platform Win32devicewatcher monitors the changes in video class devices and audio class devices through API functions registerdevicenotification. When a monitored type device sends a change, the notification is posted outward through the signaldeviceschange signal of the Devicemanagerinterface interface. Finally, the process of creating videocapturer is analyzed. DeviceManager is done through the Videodevicecapturerfactory interface when creating Videocapturer objects. The default implementation of the Videodevicecapturerfactory interface is the Webrtcvideodevicecapturerfactory class, This class creates an Webrtcvideocapturer object as an implementation of the Videocapturer interface. Can be understood as webrtcvideocapturer is the implementation of WEBRTC native video capture, but this is not accurate, because video capture involves cross-platform, not so simple. Here's a little bit more webrtcvideocapturer:

Due to platform affinity, Webrtcvideocapturer is still not the real implementation of video capture, it creates a Videocapturemodule interface object to complete the real video capture work. The abstract interface is an implementation interface for video capture, which is eventually made under the Windows platform by videocaptureds (traditional DirectShow mode) and VIDEOCAPTUREMF (the media Foundation after Vista The implementation of the API) to achieve the acquisition work. Here to illustrate the videocapturemf in the WEBRTC is still an empty shelf, has not really achieved, if the reader is interested in video collection of media Foundation API implementation, You can refer to the Videocapturedevicemfwin class implementation in Chromium's media library.

Next, analyze how Videosourceinterface and Videocapturer are combined, and who drives the acquisition to begin with.

VideoSource is the implementation of WEBRTC to Videosourceinterface interface, which holds a Videocapturer object as a video acquisition source, Videorenderer is for external to obtain video frame data from the VideoSource. In addition, VideoSource relies on the ChannelManager object, which uses the Capturemanager it contains to take charge of the video acquisition task. VideoSource will invoke the Startvideocapture method called ChannelManager in the Initialize method when it is created to begin capturing video data. Capturemanager internally maintains a capturerenderadapter for each Videocapturer object, Capturerenderadapter at the time of creation, the Onvideoframe member method hooks up the Videocapturer signalvideoframe signal to receive the video frame data transmitted by the acquisition source in real time, The Onvideoframe internally distributes the received video frame data to the Videorenderer object to which it is registered (the process of registering the Videorenderer object is VideoSource to ChannelManager, To Capturemanager, and finally to the capturerenderadapter associated with a specific videocapturer).

At this point, the implementation of Videosourceinterface in WEBRTC is very clear, the video capture process and timing is very clear, and then a simple analysis of the WEBRTC in the Videotrackinterface interface implementation:

WebRTC created a Videotrack implementation Videotrackinterface interface, before I had a question, videotrackinterface externally exposed video output interface is Videorendererinterface , while the video source interface Videosourceinterface externally exposed video output interface is Videorenderer, two sets of interfaces are how to fit. See here, I found that the original Videotrack a new Videotrackrenderers object to complete the Videorendererinterface interface to the Videorenderer interface of the adaptation work. Videotrackrenderers is derived from the Videorenderer interface on the one hand, so that it can be hooked up to receive video frame data by Videosourceinterface Addsink method. On the other hand, the received video frame data is distributed to the externally attached Videorendererinterface interface to the Videotrackinterface.

2. Chromium video capture adaptation to WebRTC

Chromium create the Webrtcvideocaptureradapter class to implement the Videocapturer interface, the related structure is as follows:

Chromium himself also encapsulated the track, source concept, so the first time to see this piece of the head is not easy to turn a lot of thought. The webrtcvideocaptureradapter needs to receive the frame data transmitted by the Chromium video capture module, which is eventually hooked up to the Mediastreamvideosource class through a layer of hooks. Mediastreamvideosource received video frame data, and then a layer of notice back, the final notification to the Webrtcvideocaptureradapter onframecaptured method, The method internally triggers the signalframecaptured signal.

The Mediastreamvideosource encapsulates the entrance to the Chromium video capture, which is complex and involves a cross-process architecture, as follows:

This section is not intended to elaborate, if the details are likely to be confused with the only few concepts established so far, this section is mainly about the Chromium WEBRTC video acquisition interface customization.

Local Video collection of WEBRTC