The combination of Kinect development Unity3d for game application development

This paper draws on the following: http://yacare.iteye.com/blog/1950164 on this basis to add a number of instructions and modifications. Recently in the development of a project with Unity3d and Kinect, suddenly found this blog, feeling that the Unity3d package is too powerful, very interesting, so recommend to everyone. To be tested by myself , I use the script in this project, after analysis, removed the redundant script, can realize the direct incoming data to control the three-dimensional model action, do not need to open the Kinect in real-time, can directly read the previous recorded bone point data, see http://blog.csdn.net/a350203223/article/details/24454805 One, unity and Kinect Interactive environment configuration

The version of Unity3d I'm using is Unity4.1, the Kinect SDK version is SDK1.7, and the way Kinect and unity interact is using the CMU package, but this method supports the Kinect SDK version is SDK1.0, to use the 1.5 and later version of the Kinect SDK, you need to make some corresponding changes. The complete scheme is as follows: 1, installation Uinty3d

The Unity3d version I use is 4.1, and I can download the free version of unity on the Unity3d website. 2. Installing the Kinect SDK

The SDK version I use is 1.7 and can be downloaded on the Kinect for Windows official website. 3. Installing the Kinect Wrapper package for Unity3d

This is the Unity3d import package from the CMU package for Kinect development. Later in the Unity3d project, if you need to interact with Kinect, you need to import this package into the project. Click here to download Kinectwrapperpackage. 4. Run the sample project

In the Kinectwrapperpackage package provided by CMU, a unity example project called Kinectexample.untiy is already included. Once you've imported the Kinectwrapperpackage package in your project, you'll find scences->kinectexample on the project panel in Unity and double-click to open the sample project.

After you open the sample project, you can see the contents in the Hierarchy panel:

5. Modify the relevant files in the import package

The Kinect SDK version supported by the Unity Import package provided by CMU is 1.0, and if you want to use a different version of the SDK, download and replace some of the files in the import package. Specifically, KinectInterop.cs and KinectSensor.cs are the two files.

Click here to download the replacement file for the Kinect SDK1.7 version.

After the download is complete, replace the downloaded file with the two files identified in the panel above. You can then run the sample project and experience using Kinect to control the motion of the character model and the ball in a scene created by unity.

Ii. resources contained in a package for Kinect development

The packages for Kinect development (Kinect Wrapper Pack for Unity3d) contain all the scripts we need to develop unity+kinect.

After the package is imported, all of the resources we need will appear in the project panel, specifically the following resources:

Scenes

Kinectexample -example scenario, this example shows you how to create a skeletal movement by placing a small ball on each of the bone points that Kinect can track. And how to use Kinect to control the model.

prefabs

Kinect_prefab-this preset (Prefab) contains all the scripts we need to develop the Unity+kinect program. But it doesn't contain the controller you need to control the model.

Kinectpointman -this preset (prefab) shows you how the kinectpointcontroller controller is used.

Kinect

KinectModelControllerV2 -you need to drag and drop this script onto the model where you want to apply Kinect control. In order for the model to be able to match the rhythm of the master, you need to drag and drop the key skeleton of the control model action on the model into the appropriate variable for the script to leak, that is, to bind the bones in the model to the bones of the person Kincet identified. In addition to the variable that this script bursts into, there is also a variable that identifies which player the model is under control.

Kinectpointcontroller -You also need to drag and drop this script onto the game object in the scene. But this game object is not a model, but consists of a series of points representing the head, shoulders, hands and other parts of the body. You need to drag and drop these key points in the game object into the external variables of the script's burst. This lets you use the Kinect to control the game object, which is made up of a series of dots.

displaydepth -This script gets the depth image.

Displaycolor -This script gets an RGB image.

Kinectrecorder -This script is used to record your actions and generate playback files for the Kinect emulator (emulator).

Kinectemulator -This script simulates the Kinect device. Works with Kinectrecorder generated playback files.

Kinectsensor -This script obtains data from the Kinect device. You need to replace this file with a special version of the SDK.

Deviceoremulator -This script is set to use the Kinect physical device or the Kinect emulation device.

Skeletonwrapper -This script captures bone data.

Depthwrapper -This script fetches the depth image data.

Kinectinterop -This script fetches data from the Microsoft Kinect SDK.

Recordings/playbackdefault -This is the default playback file that is prepared for the Kinect emulation device.

Models and Materials

Rainbowman

This is the sample model. You'll need to drag and drop the KinectModelControllerV2 script onto this model, and bind the bones that make up the model to the parts of the skeleton that are missing from the script. Then let's start the test.

third, the use of Kinectwrapperpackage package in their own projects

To use the Kinectwrapperpackage package in your own project to achieve unity3d and Kinect interaction. You first need to import this package into unity, and then follow the steps below to complete the interaction. 1. Set Kinect_prefab

As mentioned earlier, if you interact in unity and Kinect, Kinect_prefab this game object is essential. This object needs to exist somewhere in your scene. And the Dontdestroyonload function is called in the Kinectwrapper script, so when you switch the scene, it still exists, so you don't have to instantiate a kinect_prefab in every scene.

In Unity, drag the Kinect_prefab from the project panel directly onto the hierarchy panel. 2, control the role of the game

To control the movement of the characters in the scene, you need to drag the KinectModelControllerV2 script onto the model in the scene and bind the bones, or you need to drag the Kinectpointcontroller onto the object to control a series of game objects. The following describes the use of this script.

Model Controller: KinectModelControllerV2

To use the model controller, follow these steps:

Drag-and-drop script resources KinectModelControllerV2 into the model in the scene.

Select the model in the scene. Find the storm-drain variable SW in the model (it represents skeleton Wrapper). and drag the Kinect_prefab in the current scene to the SW variable.

Expand your model in detail so that each bone of the model is visible in the hierarchy panel.

Drag the skeleton from the model to the corresponding variable in the script, one by one. Make sure that each bone corresponds to the correct variable.

After all the bones in the model have been placed, change the value of the burst player variable, which indicates which player controls the model, 0 represents the first player, and 1 represents the second player.

The next step is to set the value of the mask variable for the storm drain. Set the appropriate values to determine that all bones are under Kinect control, Or just part of the skeleton is controlled by the Kinect. If these Kinect-controlled bones are not what you want, you can write a script that controls the model yourself instead of KinectModelControllerV2.

When the player controls the model, if you want the model to play its own animation at the same time, then you need to select the animated option in the burst variable and set the value of the Blendweight variable to determine the final blend of the model by its own model animation and Kinect drive action. The variable values range from 0 to 1.

For example, the corresponding relationship between bones and variables in the sample program is as follows:

Controller for a series of points:Kinectpointcontroller

In order to use the controller to control a series of points, follow these steps:

Create an empty object in the scene as a container for the series of points you want to control. Drag other objects into the empty object you just created and place them in the same position. Make the empty objects you just created become the parent nodes of these objects. Drag the Kinectpointcontroller script onto the parent node of these objects. Select the parent node, and in the Inspector panel, locate the SW variable (the variable represents skeleton Wrapper). Drag and drop the Kinect_prefab assigned to this variable in the current scene.

Drag and drop each object you want to use with Kinect control to the appropriate variable in the Inspector panel. Set the player variable (on behalf of the player) and the mask variable (which bones are monitored).

In the sample program, the correspondence between different objects and variables is as follows:

We can build our own three-dimensional model, and then bind it to the bone, in the Unity3d will be the joint point one by one, you can use unity in conjunction with Kinect developed.

The correspondence of each joint is particularly important:

Kinectwrapperpackage for SDK 1.7 replacement file Download:

http://download.csdn.net/detail/u011458382/6349895

  Kinect Development Series Blog: Kinect development Interaction Interactive design   /http yacare.iteye.com/blog/1979683   Kinect Development combined Unity3d for game application development http://yacare.iteye.com/blog/ 1950164   Kinect development body sense hands up programming (Kinect Tetris) http://yacare.iteye.com/blog/1950133   Simple posture Recognition for Kinect development http://yacare.iteye.com/blog/1950112   Kinect developed to get bone joint data and draw it into stickman. http://yacare.iteye.com/blog/1950085   Kinect developed to get color camera data   /http/ yacare.iteye.com/blog/1921786