Kinect V2 Program Design (c + +) color Chapter

Kinect SDK v2 Preview to get a description of the basic flow of data. and an introduction to the sample program that gets the color image.

The previous section is about the current model Kinect for Windows (later called Kinect v1) and the next generation of Kinect for Windows Developer Preview(later called KinectV2Preview Version) configuration comparison and introduction. start from this section，It is the comparison and introduction of the methods of obtaining the various data of Kinect. Color CameraKinect is the same as a typical web camera，equipped with a Color Camera，A color image can be obtained. About Camera，the resolution of the Kinect V1 is640x480，KinectV2significantly increased resolution to1920x1080. This time，describes how to obtain an image from color camer. the process of data acquisitionKinect SDK v1 and Kinect SDKV2the basic process of getting the data in the preview version. this represents the most basic process.，in thethe Kinect SDKV2in the preview version，have to put multiple"source""reader" API for processing，There are also differences in the process of data acquisition as described here. Figure 1 is the data acquisition process for the Kinect SDK V1 and the Kinect SDK v2 previewThe Kinect SDK V1 is a process that opens "stream" from "sensor", obtains "frame" from "stream", and obtains data from "frame". to theKinect SDKV2Preview Version，is aget "source" from "sensor"，Open "reader" from "source"，get "frame" from "reader"，the process of obtaining data from "frame". Each Kinect SDK v2 preview version of "source" has 1 streams of color and depth from Kinect, andThe Kinect SDK v1 "stream" is the same. Kinect SDK v2 Preview, with additionalthe "reader" stage. "reader" can open 1"source" several times. With this configuration，on multi-threaded applications, do not needCopy the acquired data to other threads. There are also several applications that can obtain the benefits of data from the same sensor. (note: Because the color camera and the depth camera are separate, the color, depth, and skeleton streams are separate, and the API notifies the app whether there is data updates through event.) Typically, for efficiency, a separate thread is used for each data stream to wait for the event and then process the data. In the previous V1, the frame is independent, after the data is dropped, so want to read in the depth color, such as the operation, must use the memory copy, and multi-threaded read and write data will require mutual exclusion lock, very affect efficiency;Nuisetframeendevent, the Allframeready event in the C # interface, but waiting for all the data is not only too restrictive but inefficient.) Sample ProgramThe Kinect SDK v2 preview shows a sample program of color portraits, and Figure 1 shows an excerpt from the data acquisition phase. The full contents of this sample program are made public in GitHub below.

• Https://github.com/UnaNancyOwen/Kinect2Sample

In order to process the image data, the sample program uses theOpenCV. the details of the OPENCV can be found in the following information.

• http://docs.opencv.org/

"sensor"Get "sensor"

1. //Sensorikinectsensor* Psensor; ......1HRESULT HRESULT=S_ok;hresult= Getdefaultkinectsensor (&psensor); ......2if(FAILED (HResult)) {Std::cerr<<"Error:getdefaultkinectsensor"<<Std::endl; return-1;} HResult= Psensor->open (); ......3if(FAILED (HResult)) {Std::cerr<<"Error:ikinectsensor::open ()"<<Std::endl; return-1;} 

   Listing 1.1 corresponds to Part 1 of Figure 1"sensor" processing the Sensor interface of the Kinect v2 preview. 2 Get the defaultSensor. 3 Open Sensor.

"source"
Obtained "source" from "sensor".

1. // Source icolorframesource* Pcolorsource;  ...... 1  = Psensor->get_colorframesource (&pcolorsource);  ...... 2 if (FAILED (HResult)) {  "error:ikinectsensor::get_colorframesource ()" << Std::endl;   return -1;}

   Listing 1.2equivalent to the part of Figure 1"source"1 Gets the source interface of the color frame. 2 obtained from sensorSource.

"reader""source" from Open "reader".

1. // Reader icolorframereader* Pcolorreader;  ...... 1  = Pcolorsource->openreader (&pcolorreader);  ...... 2 if (FAILED (HResult)) {  "error:icolorframesource::openreader ()" < < Std::endl;   return -1;}

   Listing 1.3 corresponds to section 1 of Figure 1"reader" to get the Reader interface of the color frame. 2 Open reader from source.

"frame"~"data"Get the latest "frame" from "reader".

1. intwidth =1920x1080; ......1intHeight = the; ......1unsignedintbuffersize = width * Height *4*sizeof(unsignedChar); ......2Cv::mat buffermat (height, width, cv_8uc4); ...3cv::mat colormat (height/2, Width/2, CV_8UC4); ......3Cv::namedwindow ("Color" ); while(1 ){  //FrameIcolorframe* pcolorframe=nullptr; ......4HResult=pcolorreader->acquirelatestframe (&pcolorframe); ......5  if(SUCCEEDED (HRESULT)) {HRESULT= Pcolorframe->copyconvertedframedatatoarray (buffersize, reinterpret_cast<byte*> (BufferMat.data),  Colorimageformat_bgra); ......6    if(SUCCEEDED (HResult)) {cv::resize (Buffermat, Colormat, Cv::size (),0.5，0.5); ......7}} saferelease (Pcolorframe); //Show WindowCv::imshow ("Color", Colormat); if(Cv::waitkey ( -) ==vk_escape) {     Break; }} 

   Listing 1.4 corresponds to the section of Figure 1"frame","data"the size of the 1 color image (1920x1080). To simplify the explanation, the image size is set by hard coding, and the sample program can get frame information from source. The data size of the 2 color image. 3in order to process the color image, prepare the OpenCV Cv::mat. "buffermat" is the original image data, "colormat" is the processing of resize image data. "cv_8uc4" is an unsigned 8bit integer (8U) with 4 channel (C4) parallel to represent 1-pixel data formats. 4 Gets the color image of the frame interface. 5 Get the latest frame from reader.   6 Get a color image from frame. The default format is YUY2 (the format of luminance and chromatic aberration), but can be transformed to Bgra by simple processing. 7 Reduce the size to half the length of the width (960x540).

When you get the data for a color image from "frame",Kinect SDK V1 is a pre-specified image size and format, the Kinect SDK v2 cannot specify an image size.  As a result, the data can be sorted in any shape after getting it. The color image that is taken out is displayed directlyThe size is too large (1920x1080)，in this case the use ofOpenCV's Resize command(Cv::resize ())，shrink to half the size of the length and width(960x540). (Note: Do not be disturbed by this section of scaling code when reading comprehension. )，just to consider the easy observation on the display，It doesn't make any sense. )Taken out ofThe data for the color image if the Bgra format is specified，will be arranged in the same way as in Figure 2. Blue(B)，Green (G)，Red (R)，and the total 32bit of the invalid value (A) constitutes 1 pixels. Figure 2 Data arrangement for a color image Run ResultsRunning the sample program will display the color image obtained from the Kinect v2 preview, as shown in Figure 3. SummaryThis section describes theKinect SDKV2basic process of getting data in preview and sample program for getting a color image，The following section describes the sample programs for obtaining depth data.

Kinect V2 Program Design (c + +) color Chapter