Use Kinect multi-point touch

It's been more than a month since we started using the alexp driver. on the Windows platform, we were able to get the camera and deep data, and then put it on a friend's Xbox for a few days, it's really powerful. I wanted to write a multi-touch software for it. To be precise, it should be an air-gap operation, like the one reported by the dispatcher. It is easy to write:

1) threshold value for deep data;

2) Blob tracking;

3) tuio sends out, and then you can enjoy various multi-touch upper-layer software.

However, considering that it is so powerful, it would be too wasteful to do so. We should take some advanced measures, such as Gesture Recognition and skeleton. However, I feel helpless, and I am not enough. It is difficult for advanced people to make their own work. Originally, this is just for entertainment, and it is hard to do so, considering the enthusiasm of international friends for cracking and sharing of open-source software, we decided to wait for good open-source software. Unfortunately, it seems that there is nothing good yet. openkinect is good, but it seems that it cannot run on Windows. Openni seems to be more powerful, but the configuration is cumbersome. I have not configured it yet... Last week, my boss urged me to do something. After all, I spent more than 1000 oceans in the lab. I couldn't pick an image every time I demonstrated it. So it took two or three days this week to figure out a software with an empty touch. In the past, I had some research on CCV when I was doing multiple touch operations, so it was very fast to change it. The effect was quite good, but it was always a bit strange to use it, it seems that we still need to be advanced in order to play the role of Kinect.

P.s. Anyone who wants to do advanced things can go to http://www.openni.org/downloadfilesto see it. If the configuration is complete, let's discuss it together ~

I just prepared a PPT for tomorrow's lab meeting, and I will share some of them here.

Kinect is a somatosensory sensor developed by Microsoft for xbox360. It is equipped with color cameras, depth sensors, acceleration sensors, and Microphone Arrays. However, due to its powerful functions and relatively low prices, geeks have expressed their desire to use it on their computers. As a result, many Daniel developed the driver. Currently, I have learned three:

1) Cl Nui Platform

Developed by Nui's Daniel alexp (other well-known products he developed include the Windows Driver of the PS3), you can download it here. The target platform is Windows 7, but it can also be used on XP, color cameras, depth sensors, and acceleration sensor data are easy to use. In my previous post, I provided a simple vs2008 project file to convert the image to the iplimage of opencv.

2) openkinect/libfreenect

It was initiated by Hector Martin, who claims to be the first to crack the Kinect. He can download it here. The target platform is Linux and Mac. It is said that someone has successfully transplanted it to Windows. The hacker did not try again... Many geek are fans of Mac, so there are a large number of developers, not just a driver that can get data, geek also wrote other advanced things, such as the skeleton I mentioned earlier, attaching a color camera as a texture to deep data is very attractive. But I cannot configure it on windows. I can only drool...

3) openni

The full name is open natural interface. You can download it here. Judging from the name, its ultimate goal is about to achieve the effect of dispatching reports. Some demos that have been released have probably surpassed this ultimate goal. It is not specifically developed for Kinect, but is supported by primesense, a maker of Kinect. The configuration is complicated, but it has not been successfully configured on Windows...

Here are some of the drivers I have learned.

Combined with the above advantages and disadvantages, as a tragedy geek who does not understand Linux and has no money to buy Mac, he can only choose Cl Nui platform. The following is a brief introduction to using this driver to rewrite CCV.

I will not introduce CCV too much. As a tool of multi-touch, almost all companies that use multi-touch products directly or indirectly learn from CCV. I am very proud to be one of the developers. So now it's easy to get started. Read the deep data and set a threshold value. If the distance is smaller than the threshold value and the remaining values are set to zero, the depth image is converted into a binary image, then, assign the image to processedimg in CCV, and then CCV findcontours, track, and send tuio as usual. To make the video look good, we also read the data from the color camera, which is displayed in the source image. It is purely for display without any processing.

Let's watch the video below. The computer is broken, and CCV, upper-layer games, and screen recording and comparison cards are enabled at the same time. In fact, the Kinect CCV can still run to 30 frames in mini mode.

Because the Code was changed from CCV, and adhering to the open-source spirit of CCV, I also put the modified part here and download it here. It contains code and compiled executable files. For students who do not know how to compile, you can directly go to the http://nuicode.com/download ccv-1.2-bin.zip, decompress and use the Community core vision.exe provided by me to overwrite the original file, use the config provided by me. XML overwrites the config. XML, and then clnuidevice under ofxkinectclnui \ DLL. copy the DLL to the directory at the same layer as community core vision.exe. Of course, you still need to install the Kinect driver first. (Because I have limited space to upload files in the blog garden, it can only help you solve the problem)

The following is a simple description.

1) install the Kinect's driver

Please note that this software currently only supports CL-NUI-Platform-1.0.0.1121 and can be downloaded here
I'm using alexp's Cl Nui platform (Version: CL-NUI-Platform-1.0.0.1121 ).

2) Check out CCV from http://nuicode.svnrepository.com/svn/tbeta/trunk/tbeta/Windows
I'm using revision 210.

3) Modify CCV following steps below:
3.1) Put ofxkinectclnui under Addons
3.2) Put clnuidevice. dll under other \ DLLs
(3.3) substitute ofxncorevision. h and ofxncorevision. cpp under Addons \ ofxncore \ SRC \ modules
(3.4) substitute config. xml under apps \ addonsexamples \ vs2008 \ bin \ data
(3.5) substitute calibration. xml under apps \ addonsexamples \ vs2008 \ bin \ data
(3.6) substitute community core vision. vcproj under apps \ addonsexamples \ vs2008

Note that the SP1 Patch must be installed for vs2008!

/* ======================********* ======================= */

In this blog post, I updated how to use the openni driver to play with multiple touch points. If you don't want to use clnuiplatform, you can try it.