OpenCV Study Notes (vii)--OPENCV for Android real-time image processing

In the previous article we have implemented the camera open and real-time image information acquisition, then we can try to get the image information to do some processing, and then real-time display, here we have to complete the processing of several:

ashing, Canny edge detection, hist histogram calculation, Sobel edge detection, SEPIA (Hue transform), Zoom Magnifier, pixelize pixelated

First, modify the layout interface:

Because here we need to switch between different image processing modes, so here we need to put a button on the interface, we can place a lot of buttons, each button corresponding to a processing mode, but here we can also only place a button, each time the button is clicked to switch, Cycle mode:

activity_main.xml File:

<framelayout xmlns:android= "http://schemas.android.com/apk/res/android" xmlns:tools= "http// Schemas.android.com/tools "xmlns:opencv=" Http://schemas.android.com/apk/res-auto "android:layout_width=" Match_ Parent "android:layout_height=" match_parent "> <org.opencv.android.javacameraview android:layout_width= "Fill_parent" android:layout_height= "fill_parent" android:id= "@+id/camera_view" opencv:show_fps= "true "opencv:camera_id=" any "/> <relativelayout android:layout_width=" Fill_parent "Android: layout_height= "Fill_parent" android:gravity= "Bottom|center_horizontal" > <button android:id = "@+id/deal_btn" android:layout_width= "100DP" android:layout_height= "40DP" android:layout _marginbottom= "20DP" android:text= "Processing"/> </RelativeLayout></FrameLayout>

To view the preview map:

Second, get the button components and listen to the button click:

1. Declare that a button object is used to bind the buttons component above and a status flag bit to store the current state:

        Button component Private button mbutton;//Current processing state private static int cur_state = 0;

2. In OnCreate, bind the button and click on the button to listen:

Mbutton = (Button) Findviewbyid (R.ID.DEAL_BTN); Mbutton.setonclicklistener (new Onclicklistener () {@Overridepublic void OnClick (View v) {if (cur_state<8) {//Toggle state cur_state + +;} else{//Restore Initial state cur_state = 0;}});

Here the status flag bit cur_state corresponds to each type of image processing, 0 corresponds to the default state, that is, the display of the original, 1-7 respectively corresponds to:ashing, Canny edge detection, hist histogram calculation, Sobel edge detection, SEPIA (Hue transform), Zoom Magnifier, pixelize pixelated

Third, image information access to save, processing and display:

1. In OpenCV, it is common to use the mat type to store matrix information such as images, so we can declare a mat object to be used as a cache object for real-time frame images:

Cache the data entered per frame of the camera private Mat Mrgba;

2. Object instantiation and settings for basic properties, including: length, width, and image type flags:

public void oncameraviewstarted (int width, int height) {//TODO auto-generated Method Stubmrgba = new Mat (height, width, C VTYPE.CV_8UC4);}

3. Object assignment, here only the original image and the ashing of the two cases are processed, the other processing later added:

        /** * Image processing is written here */@Overridepublic Mat oncameraframe (cvcameraviewframe inputframe) {switch (cur_state) {case 1:// Ashing Treatment Imgproc.cvtcolor (Inputframe.gray (), Mrgba, imgproc.color_gray2rgba,4); break;default://Display Original MRgba = Inputframe.rgba (); break;} Returns the processed result data return Mrgba;}

4. The data is stored in memory because the object is used to store the image data, so the data is released at the end, or it may cause a crash:

        @Overridepublic void oncameraviewstopped () {//TODO auto-generated method Stubmrgba.release ();}

5. Run the viewing effect:

Normal Mode:

Ashing diagram:

Iv. Other processing and results:

in the example above we have completed the grayscale processing of the preview, then we will add the other processing to the code to see the effect. Since some of the methods used in the 2.x version have changed, such as: org in OpenCV 3.1.0 . OpenCV. Core. The methods in the core class line and rectangle are invalidated and can be used with org. OpenCV. Imgproc. Imgproc line and rectangle in place of:

1. Mainactivity.java Source:

Package Com.linsh.opencv_test;import Java.util.arrays;import Org.opencv.android.baseloadercallback;import Org.opencv.android.camerabridgeviewbase;import Org.opencv.android.opencvloader;import Org.opencv.android.camerabridgeviewbase.cvcameraviewframe;import Org.opencv.android.camerabridgeviewbase.cvcameraviewlistener2;import Org.opencv.android.LoaderCallbackInterface ; Import Org.opencv.core.core;import Org.opencv.core.cvtype;import Org.opencv.core.mat;import Org.opencv.core.matoffloat;import Org.opencv.core.matofint;import Org.opencv.core.point;import Org.opencv.core.scalar;import Org.opencv.core.size;import org.opencv.imgproc.imgproc;import Android. R.string;import android.app.activity;import android.os.bundle;import Android.util.log;import Android.widget.Button ; Import Android.view.view;import Android.view.view.onclicklistener;public class Mainactivity extends Activity Implements cvcameraviewlistener2{private String TAG = "Opencv_test";//opencv Camera Interface Private Camerabridgeviewbase Mcvcamera;Cache camera input data per frame private Mat mrgba,mtmp;//button component Private button mbutton;//Current processing state private static int cur_state = 0;private Size MS    IZE0;    Private Mat Mintermediatemat;    Private Matofint mchannels[];    Private Matofint mhistsize;    private int mhistsizenum = 25;    Private Mat mMat0;    Private float[] Mbuff;    Private Matoffloat mranges;    Private point mP1;    Private point mP2;    Private Scalar mcolorsrgb[];    Private Scalar mcolorshue[];    Private Scalar Mwhilte; Private Mat msepiakernel;/** * manages Android services via OpenCV, asynchronously initializes opencv */baseloadercallback mloadercallback = new Baseloadercallback (this) {@Overridepublic void onmanagerconnected (Int. status) {switch (status) {case LOADERCALLBACKINTERFACE.SUCCESS:LOG.I (TAG, "OpenCV loaded successfully"); Mcvcamera.enableview (); Break;default: break;}}; @Overrideprotected void OnCreate (Bundle savedinstancestate) {super.oncreate (savedinstancestate); Setcontentview ( R.layout.activity_main); Mcvcamera = (camerabridgeviewbase) Findviewbyid (R.id.camera_view); Mcvcamera.setcvcameraviewlistener (this); Mbutton = (Button) Findviewbyid (R.ID.DEAL_BTN); Mbutton.setonclicklistener (New Onclicklistener () {@Overridepublic void OnClick (View v) {if (cur_state<8) {//Toggle State Cur _state + +;} else{//Restore Initial state cur_state = 0;}});} @Overridepublic void Onresume () {super.onresume (); Opencvloader.initdebug ()) {LOG.D (TAG, "OpenCV Library not found!");} else {log.d (TAG, "OpenCV Library found inside package. Using it! "); Mloadercallback.onmanagerconnected (loadercallbackinterface.success);}}; @Overridepublic void OnDestroy () {if (mcvcamera!=null) {Mcvcamera.disableview ();}};  @Overridepublic void oncameraviewstarted (int width, int height) {//TODO auto-generated Method Stubmrgba = new Mat (height,        width, cvtype.cv_8uc4); mtmp = new Mat (height, width, cvtype.cv_8uc4); Mintermediatemat = new Mat ();        MSIZE0 = new Size ();        Mchannels = new matofint[] {new Matofint (0), new Matofint (1), New Matofint (2)};        Mbuff = new Float[mhistsizenum]; Mhistsize = new Matofint (Mhistsizenum);        Mranges = new Matoffloat (0f, 256f);        MMat0 = new Mat ();        Mcolorsrgb = new scalar[] {new scalar (0, 0, 255), new scalar (0, 0, 255), new scalar (0, 0, 200, 255)}; Mcolorshue = new scalar[] {new scalar (255, 0, 0, 255), new scalar (255, 0, 255), new scalar (255, 120, 0 , 255), new scalar (255, 0, 255), new scalar (255,, 0, 255), new scalar (215, 213, 0, 255), New Scala                R (255, 0, 255), new scalar (255, 0, 255), new scalar (255, 255, 0, 255), new scalar (0, 30, 255,), New scalar (0, 255, 255), new scalar (0, 255, 255), new scalar (0, 255, 215, 255), new scalar (0, 234, 255, 255), NE W Scalar (0, 255, 255), new scalar (0, 255, 255), new scalar (0, 255, 255), new scalar (0, 0, 255 , 255), new scalar (0, 255, 255), new scalar (0, 255, 255), new scalar (0, 255, 255), new scalar (255, 0, 255, 255), New Scalar (255, 0, 215, 255), New ScAlar (255, 0, 255), New Scalar (255, 0, 0, 255)};        Mwhilte = Scalar.all (255);        mP1 = new Point ();        mP2 = new Point ();        Fill Sepia kernel Msepiakernel = new Mat (4, 4, cvtype.cv_32f);        Msepiakernel.put (0, 0,/* R */0.189f, 0.769f, 0.393f, 0f);        Msepiakernel.put (1, 0,/* G */0.168f, 0.686f, 0.349f, 0f);        Msepiakernel.put (2, 0,/* B */0.131f, 0.534f, 0.272f, 0f); Msepiakernel.put (3, 0,/* */0.000f, 0.000f, 0.000f, 1f);} @Overridepublic void oncameraviewstopped () {//TODO auto-generated method Stubmrgba.release (); Mtmp.release ();}    /** * Image processing is written here */@Overridepublic Mat oncameraframe (cvcameraviewframe inputframe) {Mrgba = Inputframe.rgba ();    Size Sizergba = Mrgba.size ();    int rows = (int) sizergba.height;    int cols = (int) sizergba.width;            Mat Rgbainnerwindow;    int left = COLS/8;    int top = ROWS/8;    int width = cols * 3/4;    int height = rows * 3/4; Switch (cur_state) {case 1://ashing processing IMGPROC.CVTColor (Inputframe.gray (), Mrgba, imgproc.color_gray2rgba,4) break;case 2://canny edge Detection Mrgba = InputFrame.rgba (); I Mgproc. Canny (Inputframe.gray (), mtmp, (+), Imgproc.cvtcolor (Mtmp, Mrgba, Imgproc.color_gray2rgba, 4); Break;case 3://            hist Histogram calculation Mat hist = new Mat ();            int thikness = (int) (Sizergba.width/(Mhistsizenum + 10)/5);            if (Thikness > 5) thikness = 5;                       int offset = (int) ((Sizergba.width-(5*mhistsizenum + 4*10) *thikness)/2); RGB for (int c=0; c<3; C + +) {imgproc.calchist (Arrays.aslist (Mrgba), Mchannels[c], MMat0, h                ist, mhistsize, mranges);                Core.normalize (hist, hist, SIZERGBA.HEIGHT/2, 0, Core.norm_inf);                Hist.get (0, 0, Mbuff); for (int h=0; h<mhistsizenum; h++) {mp1.x = mp2.x = offset + (c * (Mhistsizenum +) + h) * Thiknes                    S                    Mp1.y = sizergba.height-1;   MP2.Y = mp1.y-2-(int) mbuff[h];                 Imgproc.line (Mrgba, mP1, MP2, Mcolorsrgb[c], thikness);            }}//Value and Hue Imgproc.cvtcolor (Mrgba, mtmp, imgproc.color_rgb2hsv_full);            Value imgproc.calchist (Arrays.aslist (mtmp), mchannels[2], MMat0, Hist, Mhistsize, mranges);            Core.normalize (hist, hist, SIZERGBA.HEIGHT/2, 0, Core.norm_inf);            Hist.get (0, 0, Mbuff);                for (int h=0; h<mhistsizenum; h++) {mp1.x = mp2.x = offset + (3 * (Mhistsizenum +) + h) * thikness;                Mp1.y = sizergba.height-1;                MP2.Y = mp1.y-2-(int) mbuff[h];            Imgproc.line (Mrgba, mP1, mP2, Mwhilte, thikness);            }break;case 4://sobel Edge Detection mat Gray = Inputframe.gray ();            Mat Grayinnerwindow = Gray.submat (top, top + height, left, left + width);            Rgbainnerwindow = Mrgba.submat (top, top + height, left, left + width); Imgproc.sobel (Grayinnerwindow, MintermeDiatemat, cvtype.cv_8u, 1, 1);            Core.convertscaleabs (Mintermediatemat, Mintermediatemat, 10, 0);            Imgproc.cvtcolor (Mintermediatemat, Rgbainnerwindow, Imgproc.color_gray2bgra, 4);            Grayinnerwindow.release (); Rgbainnerwindow.release (); Break;case 5://sepia (hue transform) Rgbainnerwindow = Mrgba.submat (top, top + height, left, left + width            );            Core.transform (Rgbainnerwindow, Rgbainnerwindow, Msepiakernel); Rgbainnerwindow.release (); Break;case 6://zoom Magnifier Mat zoomcorner = Mrgba.submat (0, ROWS/2-ROWS/10, 0, COLS/2-cols            /10);  Mat Mzoomwindow = Mrgba.submat (ROWS/2-9 * rows/100, ROWS/2 + 9 * rows/100, COLS/2-9 * cols/100, COLS/2 +            9 * cols/100);            Imgproc.resize (Mzoomwindow, Zoomcorner, Zoomcorner.size ());            Size wsize = Mzoomwindow.size (); Imgproc.rectangle (Mzoomwindow, New Point (1, 1), New Point (Wsize.width-2, wsize.height-2), new Scalar (255, 0, 0, 255),            2);Zoomcorner.release ();            Mzoomwindow.release (); break;case 7://pixelize pixelated Rgbainnerwindow = Mrgba.submat (top, top + height, left, left + width);            Imgproc.resize (Rgbainnerwindow, Mintermediatemat, MSize0, 0.1, 0.1, imgproc.inter_nearest);            Imgproc.resize (Mintermediatemat, Rgbainnerwindow, Rgbainnerwindow.size (), 0., 0., imgproc.inter_nearest); Rgbainnerwindow.release (); break;default://Display Original Mrgba = Inputframe.rgba (); break;} Returns the processed result data return Mrgba;}}

2.:

Canny edge detection:

hist Histogram calculation:

Sobel Edge Detection:

SEPIA (Hue Transform):

Zoom Magnifier:

Pixelize pixelated:

OpenCV Study Notes (vii)--OPENCV for Android real-time image processing