WebRTC學習之九：網路攝影機的捕捉和顯示

標籤：sso   機制   sig   file   open   push   sdn   mic   roc   

           較新的WebRTC源碼中已經沒有了與VoiceEngine結構對應的VidoeEngine了，取而代之的是MeidaEngine。MediaEngine包含了MediaEngineInterface介面及其實現CompositeMediaEngine，CompositeMediaEngine本身也是個模板類，兩個模板參數分別是音頻引擎和視頻引擎。CompositeMediaEngine衍生類別WebRtcMediaEngine依賴的模板參數是WebRtcVoiceEngine和WebRtcVideoEngine2。
        中base目錄中是一些抽象類別，engine目錄中是對應抽象類別的實現，使用時直接調用engine目錄中的介面即可。WebRtcVoiceEngine實際上是VoiceEngine的再次封裝，它使用VoiceEngine進行音頻處理。注意命名，WebRtcVideoEngine2帶了個2字，不用想，這肯定是個升級版本的VideoEngine，還有個WebRtcVideoEngine類。WebRtcVideoEngine2比WebRtcVideoEngine改進之處在於將視頻流一分為二：發送流（WebRtcVideoSendStream）和接收流（WebRtcVideoReceiveStream），從而結構上更合理，源碼更清晰。
       本文的實現主要是使用了WebRtcVideoEngine2中WebRtcVideoCapturer類。
一.環境
參考上篇：WebRTC學習之三：錄音和播放
二.實現
        開啟WebRtcVideoCapturer的標頭檔webrtcvideocapture.h，公有的函數基本上都是base目錄中VideoCapturer類的實現，用於初始化裝置和啟動捕捉。私人函數OnIncomingCapturedFrame和OnCaptureDelayChanged會在網路攝影機採集模組VideoCaptureModeule中回調，將採集的映像傳給OnIncomingCapturedFrame，並將採集的延時變化傳給OnCaptureDelayChanged。
        WebRTC中也實現了類似Qt中的訊號和槽機制，詳見WebRTC學習之七：精鍊的訊號和槽機制 。但是就像在該文中提到的，sigslot.h中的emit函數名會和Qt中的emit宏衝突，我將sigslot.h中的emit改成了Emit，當然改完之後，需要重新編譯rtc_base工程。
       VideoCapturer類有兩個訊號sigslot::signal2<VideoCapturer*, CaptureState> SignalStateChange和sigslot::signal2<VideoCapturer*, const CapturedFrame*, sigslot::multi_threaded_local> SignalFrameCaptured，從SignalFrameCaptured的參數可以看出我們只要實現對應的槽函數就能擷取到CapturedFrame，在槽函數中將 CapturedFrame進行轉換顯示即可。SignalStateChange訊號的參數CaptureState是個枚舉，標識捕捉的狀態（停止、開始、進行中、失敗）。
        訊號SignalFrameCaptured正是在回呼函數OnIncomingCapturedFrame中發射出去的。OnIncomingCapturedFrame裡面用到了函數的非同步執行，詳見WebRTC學習之八：函數的非同步執行。

mainwindow.h

#ifndef MAINWINDOW_H#define MAINWINDOW_H#include <QMainWindow>#include <QDebug>#include <map>#include <memory>#include <string>#include "webrtc/base/sigslot.h"#include "webrtc/modules/video_capture/video_capture.h"#include "webrtc/modules/video_capture/video_capture_factory.h"#include "webrtc/media/base/videocapturer.h"#include "webrtc/media/engine/webrtcvideocapturer.h"#include "webrtc/media/engine/webrtcvideoframe.h"namespace Ui {class MainWindow;}class MainWindow : public QMainWindow,public sigslot::has_slots<>{    Q_OBJECTpublic:    explicit MainWindow(QWidget *parent = 0);    ~MainWindow();    void OnFrameCaptured(cricket::VideoCapturer* capturer, const cricket::CapturedFrame* frame);    void OnStateChange(cricket::VideoCapturer* capturer, cricket::CaptureState state);private slots:    void on_pushButtonOpen_clicked();private:     void getDeviceList();private:    Ui::MainWindow *ui;    cricket::WebRtcVideoCapturer *videoCapturer;    cricket::WebRtcVideoFrame *videoFrame;    std::unique_ptr<uint8_t[]> videoImage;    QStringList deviceNameList;    QStringList deviceIDList;};#endif // MAINWINDOW_H

mainwindow.cpp

#include "mainwindow.h"#include "ui_mainwindow.h"MainWindow::MainWindow(QWidget *parent) :    QMainWindow(parent),    ui(new Ui::MainWindow),    videoCapturer(new cricket::WebRtcVideoCapturer()),    videoFrame(new cricket::WebRtcVideoFrame()){   ui->setupUi(this);   getDeviceList();}MainWindow::~MainWindow(){    delete ui;    videoCapturer->SignalFrameCaptured.disconnect(this);    videoCapturer->SignalStateChange.disconnect(this);    videoCapturer->Stop();}void MainWindow::OnFrameCaptured(cricket::VideoCapturer* capturer,const cricket::CapturedFrame* frame){    videoFrame->Init(frame, frame->width, frame->height,true);    //將視頻映像轉成RGB格式    videoFrame->ConvertToRgbBuffer(cricket::FOURCC_ARGB,                                  videoImage.get(),                                  videoFrame->width()*videoFrame->height()*32/8,                                  videoFrame->width()*32/8);    QImage image(videoImage.get(), videoFrame->width(), videoFrame->height(), QImage::Format_RGB32);    ui->label->setPixmap(QPixmap::fromImage(image));}void MainWindow::OnStateChange(cricket::VideoCapturer* capturer, cricket::CaptureState state){}void MainWindow::getDeviceList(){    deviceNameList.clear();    deviceIDList.clear();    webrtc::VideoCaptureModule::DeviceInfo *info=webrtc::VideoCaptureFactory::CreateDeviceInfo(0);    int deviceNum=info->NumberOfDevices();    for (int i = 0; i < deviceNum; ++i)    {        const uint32_t kSize = 256;        char name[kSize] = {0};        char id[kSize] = {0};        if (info->GetDeviceName(i, name, kSize, id, kSize) != -1)        {            deviceNameList.append(QString(name));            deviceIDList.append(QString(id));            ui->comboBoxDeviceList->addItem(QString(name));        }    }    if(deviceNum==0)    {        ui->pushButtonOpen->setEnabled(false);    }}void MainWindow::on_pushButtonOpen_clicked(){    static bool flag=true;    if(flag)    {         ui->pushButtonOpen->setText(QStringLiteral("關閉"));        const std::string kDeviceName = ui->comboBoxDeviceList->currentText().toStdString();        const std::string kDeviceId = deviceIDList.at(ui->comboBoxDeviceList->currentIndex()).toStdString();        videoCapturer->Init(cricket::Device(kDeviceName, kDeviceId));        int width=videoCapturer->GetSupportedFormats()->at(0).width;        int height=videoCapturer->GetSupportedFormats()->at(0).height;        cricket::VideoFormat format(videoCapturer->GetSupportedFormats()->at(0));        //開始捕捉        if(cricket::CS_STARTING == videoCapturer->Start(format))        {            qDebug()<<"Capture is started";        }        //串連WebRTC的訊號和槽        videoCapturer->SignalFrameCaptured.connect(this,&MainWindow::OnFrameCaptured);        videoCapturer->SignalStateChange.connect(this,&MainWindow::OnStateChange);        if(videoCapturer->IsRunning())        {            qDebug()<<"Capture is running";        }        videoImage.reset(new uint8_t[width*height*32/8]);    }    else    {        ui->pushButtonOpen->setText(QStringLiteral("開啟"));        //重複串連會報錯，需要先斷開，才能再次串連        videoCapturer->SignalFrameCaptured.disconnect(this);        videoCapturer->SignalStateChange.disconnect(this);        videoCapturer->Stop();        if(!videoCapturer->IsRunning())        {            qDebug()<<"Capture is stoped";        }        ui->label->clear();    }    flag=!flag;}

main.cpp

#include "mainwindow.h"#include <QApplication>int main(int argc, char *argv[]){    QApplication a(argc, argv);    MainWindow w;    w.show();    while(true)    {        //WebRTC訊息迴圈        rtc::Thread::Current()->ProcessMessages(0);        rtc::Thread::Current()->SleepMs(1);        //Qt訊息迴圈        a.processEvents( );    }}

注意main函數中對WebRTC和Qt訊息迴圈的處理，這是用Qt調用WebRTC進行相機擷取和顯示的關鍵。

三.效果

WebRTC學習之九：網路攝影機的捕捉和顯示