Code introduction to Kinect Bridge OpenCV Kinect Bridge With OpenCV

Introduction to the code of Kinect bridge OpenCV KinectBridge With OpenCV

 Kinect
Bridge With MATLAB & OpenCV profile and project download

Driver version: Kinect forWindows SDK v1.7

Project name: KinectBridgeWithOpenCVBasics-D2D

Programming Language: C ++

Program configuration and running

Documentation. This document only describes how to configure OpenCV environment variables and so on. We can understand it at a Glance using OpenCV's child paper. Here I still use my usual method to configure. (If OpenCV has not been configured, go to opencv.org.cn to learn how to configure it. It usually takes 1 hour to 1 day for beginners to learn how to configure it. 5 minutes after being proficient) [You can also configure it according to the method described by Microsoft. Although Microsoft's method is troublesome, it should be done once and for all!]

 

I installed OpenCV2.4.4 IN THE Softs folder of the F disk. Use VS2010 for configuration.

On the properties page, ---> VC ++ directory:

Include directory:

F: \ Softs \ OpenCV244 \ opencv \ build \ include

$ (KINECTSDK10_DIR) \ inc

Library directory:

F: \ Softs \ OpenCV244 \ opencv \ build \ x86 \ vc10 \ lib

$ (KINECTSDK10_DIR) \ lib \ x86

Source directory:

F: \ Softs \ OpenCV244 \ opencv \ modules \ core \ src

F: \ Softs \ OpenCV244 \ opencv \ modules \ imgproc \ src

F: \ Softs \ OpenCV244 \ opencv \ modules \ highgui \ src

On the properties page ---> linker:

Additional dependencies:

Opencv_core244d.lib

Opencv_imgproc244d.lib

Opencv_highgui244d.lib

Kinect10.lib

Comctl32.lib

 

After all the configurations are complete, compile F7 and then execute Ctrl + F5. There is no problem at all (if there is a problem with your configuration, don't leave a message, please go to the OpenCV forum for help) and get the following running image (my portrait has been brushed off with a paint brush ):

Brief program explanation

FrameRateTracker class

This function is used to calculate the video frame rate. With the clock () function, the frame rate display result is adjusted once per second.

KinectHelper class

The definition and implementation of this class are included in the header file. The main function is to call the functions in the Kinect SDK to initialize the Kinect (enable the Kinect and various data streams, update the color and depth image data frames and the skeleton data frames) and synchronize the event objects. Data is stored in the m_pColorBuffer, m_pDepthBuffer, and m_skeletonFrame arrays.

OpenCVHelper class

This method is used to filter images (Gaussian Blur GaussianBlur, corrode erode, swell dilate, and canndy edge detection) and slide the skeleton data to color images and Deep Images (line and circle methods ).

OpenCVFrameHelper class

The subclass of KinectHelper stores data in the Mat matrix structure of OpenCV (method pImage-> ptr <Vec4b> (y )).

========================================================== ======================================

MainWindow class

Main program. The CreateMutex function is used for synchronization (their previous programs are not synchronized). The CreateMutex () function can be used to create a known or unknown mutex object.

HANDLE CreateMutex (

LPSECURITY_ATTRIBUTES lpMutexAttributes, // pointer to the Security Attribute

BOOL bInitialOwner, // initialize the owner of the mutex object

Lptstr lpName // pointer to the mutex object name );

 

Return Value

Long. If the execution is successful, the handle of the mutex object is returned. Zero indicates an error. GetLastError is set. Even if a valid handle is returned, if the specified name already exists, GetLastError is set to ERROR_ALREADY_EXISTS.

 

Parameter table parameter types and descriptions

LpMutexAttributes SECURITY_ATTRIBUTES, Specify a SECURITY_ATTRIBUTES structure, or pass the zero value (declare the parameter As ByVal As Long, and pass the zero value ),

Indicates that the default descriptor that cannot be inherited is used.

 

BInitialOwner BOOLIf the creation process wants to have a mutex immediately, set it to TRUE. A mutex can only be owned by one thread. FALSE indicates that the created Mutex does not belong to any thread.

That is to say, no thread owns him. When a Mutex has no thread, he is in an exciting state, so he is in a signal state.

 

LpName StringSpecifies the name of the mutex object. Use vbNullString to create an unnamed mutex object. If an event with this name already exists, open the existing named mutex. This name may not match the existing event, signal, wait timer, or file ing. This name may have a "Global \" or "Local \" prefix, explicitly create an object in the global or session namespace. The remaining names can contain any character except the backslash (\).

 

Generation Method:

M_hColorResolutionMutex = CreateMutex (NULL, FALSE, NULL );

All mutex lock parameters created by Microsoft are the same. It indicates that 1 uses the default descriptor that cannot be inherited, 2 the Mutex just created does not belong to any thread, and 3 the Mutex object has no name.

Call method:

WaitForSingleObject (m_hColorResolutionMutex, INFINITE );

M_colorResolution = NUI_IMAGE_RESOLUTION_640x480;

ReleaseMutex (m_hColorResolutionMutex );

The method is the same as that in the critical section. These Synchronization Methods (mutex lock, critical section) are used because some variables (or resources) are used in multiple threads at the same time. To avoid resource usage conflicts, this variable is used by multiple threads in queue. All variables used in multiple threads at the same time need to set a synchronization method for them (if there are too many variables, it is best to combine them into a variable structure ).

 

 

Thread start (in the Run function ):

M_hProcessThread = CreateThread (NULL, 0, ProcessThread,
This, 0, NULL );

Thread end event:

M_hProcessStopEvent = CreateEvent (NULL, FALSE, FALSE, NULL );

 

Use the while infinite loop in the main thread ProcessThread

Four event objects (426 rows)

HANDLE hEvents [4] = {m_hProcessStopEvent (exit), color, depth, skeleton };

Wait for a kernel object to be triggered (four in total)

Int eventId = WaitForMultipleObjects (numEvents, hEvents, FALSE, 100 );

Then, it will be processed based on different situations. Except for the exit (eventId = 0), all other data is directly updated, but not processed separately for the eventId size (, 3. This is an improvement. Other C ++ programs have all been changed to this type. It should be easy to calculate. However, this will cause only one function to run at a time, and the other two will not run. If
(WAIT_OBJECT_0 = WaitForSingleObject (m_hNextDepthFrameEvent, 0) are all removed. The program can run the same way, but the system will prompt that the data is not captured (the three data cannot be obtained at the same time and must be queued ).

 

In MainWindow, you can also see how the Mat of OpenCV is used:

1. Raw Data BYTE --> stored in m_pColorBuffer (m_frameHelper.UpdateColorFrame ())

2. m_pColorBuffer is constructed into m_colorMat structure (m_frameHelper.GetColorImage (& m_colorMat). 3. Various filtering methods for m_colorMat (m_openCVHelper.ApplyColorFilter (& m_colorMat ))

4. finally, the filtered image (UpdateBitmap (& m_colorMat, & m_hColorBitmap, & m_bmiColor); the underlying layer calls SetDIBits (m_hdc, * phBitmap, 0, height, pImg-> ptr (), pBmi, DIB_RGB_COLORS );)

5. At the end, you need to refresh the screen area (InvalidateRect (m_hWndMain, NULL, false );)

 

Using windows APIs to create a form makes the program very bloated. There are a total of 3000 lines of code (including comments and empty lines ). But there are still many things to learn, especially the code they write is exquisite. This program only uploads data to the OpenCV structure, and uses the basic functions of OpenCV for some image processing. Finally, it is displayed that this principle is implemented by the Article code written more than a year ago.

If you have been using windows api to display windows, you can easily get started. If not, it will take a lot of time to learn it (I have been using simple MFC as a window ). The examples provided by Microsoft are robust and stable, and many situations have been taken into consideration. But if you want to implement better software on your own, we suggest using MFC.

You can directly make some changes in my previous article, Kinect Face Tracking, And the Kinect Face Tracking SDK, to get the image displayed by OpenCV. I don't know how many people use MFC. If there are too many people, I can take the time to write a simple example. I Don't Know If Qt (developed in VS2010) can be displayed.