OpenCV學習筆記（七）—— OpenCV for Android即時影像處理

標籤：

        在上篇中我們已經實現了相機開啟和即時映像資訊的擷取，那麼接下來我們可以嘗試在擷取的映像資訊進行一些處理，然後即時顯示出來，在這裡我們要完成的的幾種處理：

        灰化、Canny邊緣檢測、Hist長條圖計算、Sobel邊緣檢測、SEPIA(色調變換)、ZOOM放大鏡、PIXELIZE像素化

一、修改布局介面：

        由於這裡我們需要切換不同的影像處理模式，所以這裡我們需要在介面上放置一個按鈕，我們可以放置很多個按鈕，每個按鈕對應一種處理模式，但是這裡我們也可以只放置一個按鈕，每次點擊按鈕就切換一次，迴圈切換模式：

        activity_main.xml檔案：

<FrameLayout     xmlns:android="http://schemas.android.com/apk/res/android"    xmlns:tools="http://schemas.android.com/tools"    xmlns:opencv="http://schemas.android.com/apk/res-auto"    android:layout_width="match_parent"    android:layout_height="match_parent">    <org.opencv.android.JavaCameraView         android:layout_width="fill_parent"        android:layout_height="fill_parent"        android:id="@+id/camera_view"        opencv:show_fps="true"         opencv:camera_id="any"/>        <RelativeLayout         android:layout_width="fill_parent"        android:layout_height="fill_parent"        android:gravity="bottom|center_horizontal">        <Button             android:id="@+id/deal_btn"            android:layout_width="100dp"            android:layout_height="40dp"            android:layout_marginBottom="20dp"            android:text="處理"/>    </RelativeLayout></FrameLayout>

        查看預覽圖：
        

二、擷取按鈕組件並監聽按鈕點擊：

1.聲明一個Button對象用於綁定上面的按鈕組件和一個狀態標誌位用於儲存目前狀態：

        //按鈕組件private Button mButton;//當前處理狀態private static int Cur_State = 0;

2.在OnCreate中綁定按鈕和按鈕點擊監聽：

mButton = (Button) findViewById(R.id.deal_btn);mButton.setOnClickListener(new OnClickListener(){@Overridepublic void onClick(View v) {if(Cur_State<8){//切換狀態Cur_State ++;}else{//恢複初始狀態Cur_State = 0;}}});

        這裡的狀態標誌位Cur_State與影像處理的每個類型對應，0對應預設狀態，也就是顯示原圖，1-7分別對應：灰化、Canny邊緣檢測、Hist長條圖計算、Sobel邊緣檢測、SEPIA(色調變換)、ZOOM放大鏡、PIXELIZE像素化

三、映像資訊擷取儲存、處理和顯示：

1.在OpenCV中一般都是使用Mat類型來儲存映像等矩陣資訊，所以我們可以聲明一個Mat對象用來作為即時幀映像的緩衝對象：

//緩衝相機每幀輸入的資料private Mat mRgba;

2.對象執行個體化以及基本屬性的設定，包括：長度、寬度和映像類型標誌：

public void onCameraViewStarted(int width, int height) {// TODO Auto-generated method stubmRgba = new Mat(height, width, CvType.CV_8UC4);}

3.對象賦值，這裡只對原圖和灰化兩種情況進行了處理，其他的處理後續再添加：

        /** * 影像處理都寫在此處 */@Overridepublic Mat onCameraFrame(CvCameraViewFrame inputFrame) {switch (Cur_State) {case 1://灰化處理Imgproc.cvtColor(inputFrame.gray(), mRgba, Imgproc.COLOR_GRAY2RGBA,4);break;default://顯示原圖mRgba = inputFrame.rgba();break;}//返回處理後的結果資料return mRgba;}

4.由於用Object Storage Service映像資料的話，資料會儲存到記憶體中，所以結束的時候需要進行資料釋放，不然可能導致崩潰：

        @Overridepublic void onCameraViewStopped() {// TODO Auto-generated method stubmRgba.release();}

5.運行查看效果：

        正常模式：

        
        灰化圖：

        

四、其他處理及結果：

        在以上的例子中我們已經完成了預覽圖的灰化處理，那麼接下來我們把其他處理都添加到代碼中，查看效果。由於在2.x版本中使用到的部分方法已經發生了變化，如：在OpenCV 3.1.0中org.opencv.core.Core類中的方法line和rectangle都已失效，可以用org.opencv.imgproc.Imgproc中的line和rectangle來代替：

1. MainActivity.java源碼：

package com.linsh.opencv_test;import java.util.Arrays;import org.opencv.android.BaseLoaderCallback;import org.opencv.android.CameraBridgeViewBase;import org.opencv.android.OpenCVLoader;import org.opencv.android.CameraBridgeViewBase.CvCameraViewFrame;import org.opencv.android.CameraBridgeViewBase.CvCameraViewListener2;import org.opencv.android.LoaderCallbackInterface;import org.opencv.core.Core;import org.opencv.core.CvType;import org.opencv.core.Mat;import org.opencv.core.MatOfFloat;import org.opencv.core.MatOfInt;import org.opencv.core.Point;import org.opencv.core.Scalar;import org.opencv.core.Size;import org.opencv.imgproc.Imgproc;import android.R.string;import android.app.Activity;import android.os.Bundle;import android.util.Log;import android.widget.Button;import android.view.View;import android.view.View.OnClickListener;public class MainActivity extends Activity implements CvCameraViewListener2{private String TAG = "OpenCV_Test";//OpenCV的相機介面private CameraBridgeViewBase mCVCamera;//緩衝相機每幀輸入的資料private Mat mRgba,mTmp;//按鈕組件private Button mButton;//當前處理狀態private static int Cur_State = 0;private Size mSize0;    private Mat mIntermediateMat;    private MatOfInt mChannels[];    private MatOfInt mHistSize;    private int mHistSizeNum = 25;    private Mat mMat0;    private float[] mBuff;    private MatOfFloat mRanges;    private Point mP1;    private Point mP2;    private Scalar mColorsRGB[];    private Scalar mColorsHue[];    private Scalar mWhilte;    private Mat mSepiaKernel;/** * 通過OpenCV管理Android服務，非同步初始化OpenCV */BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {@Overridepublic void onManagerConnected(int status){switch (status) {case LoaderCallbackInterface.SUCCESS:Log.i(TAG,"OpenCV loaded successfully");mCVCamera.enableView();break;default:break;}}};@Overrideprotected void onCreate(Bundle savedInstanceState) {super.onCreate(savedInstanceState);setContentView(R.layout.activity_main);mCVCamera = (CameraBridgeViewBase) findViewById(R.id.camera_view);mCVCamera.setCvCameraViewListener(this);mButton = (Button) findViewById(R.id.deal_btn);mButton.setOnClickListener(new OnClickListener(){@Overridepublic void onClick(View v) {if(Cur_State<8){//切換狀態Cur_State ++;}else{//恢複初始狀態Cur_State = 0;}}});}@Overridepublic void onResume() {super.onResume();if (!OpenCVLoader.initDebug()) {Log.d(TAG,"OpenCV library not found!");} else {Log.d(TAG, "OpenCV library found inside package. Using it!");mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);}};@Overridepublic void onDestroy() {if(mCVCamera!=null){mCVCamera.disableView();}};@Overridepublic void onCameraViewStarted(int width, int height) {// TODO Auto-generated method stubmRgba = new Mat(height, width, CvType.CV_8UC4);mTmp = new Mat(height, width, CvType.CV_8UC4);mIntermediateMat = new Mat();        mSize0 = new Size();        mChannels = new MatOfInt[] { new MatOfInt(0), new MatOfInt(1), new MatOfInt(2) };        mBuff = new float[mHistSizeNum];        mHistSize = new MatOfInt(mHistSizeNum);        mRanges = new MatOfFloat(0f, 256f);        mMat0 = new Mat();        mColorsRGB = new Scalar[] { new Scalar(200, 0, 0, 255), new Scalar(0, 200, 0, 255), new Scalar(0, 0, 200, 255) };        mColorsHue = new Scalar[] {                new Scalar(255, 0, 0, 255), new Scalar(255, 60, 0, 255), new Scalar(255, 120, 0, 255), new Scalar(255, 180, 0, 255), new Scalar(255, 240, 0, 255),                new Scalar(215, 213, 0, 255), new Scalar(150, 255, 0, 255), new Scalar(85, 255, 0, 255), new Scalar(20, 255, 0, 255), new Scalar(0, 255, 30, 255),                new Scalar(0, 255, 85, 255), new Scalar(0, 255, 150, 255), new Scalar(0, 255, 215, 255), new Scalar(0, 234, 255, 255), new Scalar(0, 170, 255, 255),                new Scalar(0, 120, 255, 255), new Scalar(0, 60, 255, 255), new Scalar(0, 0, 255, 255), new Scalar(64, 0, 255, 255), new Scalar(120, 0, 255, 255),                new Scalar(180, 0, 255, 255), new Scalar(255, 0, 255, 255), new Scalar(255, 0, 215, 255), new Scalar(255, 0, 85, 255), new Scalar(255, 0, 0, 255)        };        mWhilte = Scalar.all(255);        mP1 = new Point();        mP2 = new Point();        // Fill sepia kernel        mSepiaKernel = new Mat(4, 4, CvType.CV_32F);        mSepiaKernel.put(0, 0, /* R */0.189f, 0.769f, 0.393f, 0f);        mSepiaKernel.put(1, 0, /* G */0.168f, 0.686f, 0.349f, 0f);        mSepiaKernel.put(2, 0, /* B */0.131f, 0.534f, 0.272f, 0f);        mSepiaKernel.put(3, 0, /* A */0.000f, 0.000f, 0.000f, 1f);}@Overridepublic void onCameraViewStopped() {// TODO Auto-generated method stubmRgba.release();mTmp.release();}/** * 影像處理都寫在此處 */@Overridepublic Mat onCameraFrame(CvCameraViewFrame inputFrame) {mRgba = inputFrame.rgba();    Size sizeRgba = mRgba.size();    int rows = (int) sizeRgba.height;    int cols = (int) sizeRgba.width;    Mat rgbaInnerWindow;            int left = cols / 8;    int top = rows / 8;    int width = cols * 3 / 4;    int height = rows * 3 / 4;    switch (Cur_State) {case 1://灰化處理Imgproc.cvtColor(inputFrame.gray(), mRgba, Imgproc.COLOR_GRAY2RGBA,4);break;case 2://Canny邊緣檢測mRgba = inputFrame.rgba();Imgproc.Canny(inputFrame.gray(), mTmp, 80, 100);Imgproc.cvtColor(mTmp, mRgba, Imgproc.COLOR_GRAY2RGBA, 4);break;case 3://Hist長條圖計算Mat hist = new Mat();            int thikness = (int) (sizeRgba.width / (mHistSizeNum + 10) / 5);            if(thikness > 5) thikness = 5;            int offset = (int) ((sizeRgba.width - (5*mHistSizeNum + 4*10)*thikness)/2);                       // RGB            for(int c=0; c<3; c++) {                Imgproc.calcHist(Arrays.asList(mRgba), mChannels[c], mMat0, hist, mHistSize, mRanges);                Core.normalize(hist, hist, sizeRgba.height/2, 0, Core.NORM_INF);                hist.get(0, 0, mBuff);                for(int h=0; h<mHistSizeNum; h++) {                    mP1.x = mP2.x = offset + (c * (mHistSizeNum + 10) + h) * thikness;                    mP1.y = sizeRgba.height-1;                    mP2.y = mP1.y - 2 - (int)mBuff[h];                    Imgproc.line(mRgba, mP1, mP2, mColorsRGB[c], thikness);                }            }            // Value and Hue            Imgproc.cvtColor(mRgba, mTmp, Imgproc.COLOR_RGB2HSV_FULL);            // Value            Imgproc.calcHist(Arrays.asList(mTmp), mChannels[2], mMat0, hist, mHistSize, mRanges);            Core.normalize(hist, hist, sizeRgba.height/2, 0, Core.NORM_INF);            hist.get(0, 0, mBuff);            for(int h=0; h<mHistSizeNum; h++) {                mP1.x = mP2.x = offset + (3 * (mHistSizeNum + 10) + h) * thikness;                mP1.y = sizeRgba.height-1;                mP2.y = mP1.y - 2 - (int)mBuff[h];                Imgproc.line(mRgba, mP1, mP2, mWhilte, thikness);            }break;case 4://Sobel邊緣檢測Mat gray = inputFrame.gray();            Mat grayInnerWindow = gray.submat(top, top + height, left, left + width);            rgbaInnerWindow = mRgba.submat(top, top + height, left, left + width);            Imgproc.Sobel(grayInnerWindow, mIntermediateMat, CvType.CV_8U, 1, 1);            Core.convertScaleAbs(mIntermediateMat, mIntermediateMat, 10, 0);            Imgproc.cvtColor(mIntermediateMat, rgbaInnerWindow, Imgproc.COLOR_GRAY2BGRA, 4);            grayInnerWindow.release();            rgbaInnerWindow.release();break;case 5://SEPIA(色調變換)rgbaInnerWindow = mRgba.submat(top, top + height, left, left + width);            Core.transform(rgbaInnerWindow, rgbaInnerWindow, mSepiaKernel);            rgbaInnerWindow.release();break;case 6://ZOOM放大鏡Mat zoomCorner = mRgba.submat(0, rows / 2 - rows / 10, 0, cols / 2 - cols / 10);            Mat mZoomWindow = mRgba.submat(rows / 2 - 9 * rows / 100, rows / 2 + 9 * rows / 100, cols / 2 - 9 * cols / 100, cols / 2 + 9 * cols / 100);            Imgproc.resize(mZoomWindow, zoomCorner, zoomCorner.size());            Size wsize = mZoomWindow.size();            Imgproc.rectangle(mZoomWindow, new Point(1, 1), new Point(wsize.width - 2, wsize.height - 2), new Scalar(255, 0, 0, 255), 2);            zoomCorner.release();            mZoomWindow.release();break;case 7://PIXELIZE像素化rgbaInnerWindow = mRgba.submat(top, top + height, left, left + width);            Imgproc.resize(rgbaInnerWindow, mIntermediateMat, mSize0, 0.1, 0.1, Imgproc.INTER_NEAREST);            Imgproc.resize(mIntermediateMat, rgbaInnerWindow, rgbaInnerWindow.size(), 0., 0., Imgproc.INTER_NEAREST);            rgbaInnerWindow.release();break;default://顯示原圖mRgba = inputFrame.rgba();break;}//返回處理後的結果資料return mRgba;}}

2.：

Canny邊緣檢測：

        

Hist長條圖計算：

        

Sobel邊緣檢測：

        

SEPIA(色調變換)：

        

ZOOM放大鏡：

        

PIXELIZE像素化：
        

OpenCV學習筆記（七）—— OpenCV for Android即時影像處理