Kinect for Windows SDK development entry (18): Kinect Interaction Interactive Control

Version 1.7 of the SDK was released in March this year. The biggest change in this version of the SDK over the previous version is the addition of Kinect Interactions and Kinect Fusion. Kinect Interactions provides some new controls with gesture recognition such as push-to-press buttons, grip-to-pan list controls, and supports multi-user, two-person interaction at the same time. These newly added controls can be very convenient Integration into the application greatly simplifies the development and debugging process.

First, establish the necessary environment

    Before creating the project, you need to download the latest 1.7 SDK and Developer Toolkit from the official website. First open Visual Studio to create a simple WPF desktop application, and then add Microsoft.Kinect.dll, Kinect.Toolkit.dll, Kinect.Toolkit.Controls.dll and Kinect.Toolkit.Interaction.dll references, these dlls are generally in the installation directory Next, my machine is under C: \ Program Files \ Microsoft SDKs \ Kinect \.

Initialize the Kinect sensor using the KinectSensorChooser control

    Under the Microsoft.Kinect.Toolkit.Controls namespace, the first control used may be the KinectSensorChooserUI, which is used to indicate the current Kinect working status, and prompts the user if the Kinect sensor is working properly, such as whether it is disconnected or inserted into the wrong USB interface and so on.

   To add this control, first add the following namespace in the main form:

   Then, add the KinectSensorChooserUI control in the main form, the code is as follows:

   In the cs code, we need to initialize the KinectSensorChooser control object:

sensorChooser;
   Then register the OnLoad event in the constructor of the main form:

MainWindow ()
{
    InitializeComponent ();
    Loaded + = OnLoaded;
}
   Create OnLoaded delegate method:

OnLoaded (sender, e)
{
    .sensorChooser = ();
    .sensorChooser.KinectChanged + = SensorChooserOnKinectChanged;
    .sensorChooserUi.KinectSensorChooser = .sensorChooser;
    .sensorChooser.Start ();
}
    If the state of the sensor changes, such as shutdown or initialization is complete, the SensorChooserOnKinectChanged event will be triggered. Now for the convenience of demonstration, use MessageBox to pop up the prompt message:

SensorChooserOnKinectChanged (sender, args)
{
    .Show (args.NewSensor ==?: Args.NewSensor.Status.ToString ());
}
    Run the system now. If Kinect is not connected to your computer, you will see:

SensorChooserOnKinectChanged (sender, args) {error =; (args.OldSensor! =) {{Args.OldSensor.DepthStream.Range = .Default; args.OldSensor.SkeletonStream.EnableTrackingInNearRange =; args.OldSensor.DepthStream.Disable (); .OldSensor.SkeletonStream.Disable ();} () {error =;}} (args.NewSensor! =) {{Args.NewSensor.DepthStream.Enable (.Resolution640x480Fps30); args.NewSensor.SkeletonStream.Enable (); { args.NewSensor.DepthStream.Range = .Near; args.NewSensor.SkeletonStream.EnableTrackingInNearRange =; args.NewSensor.SkeletonStream.TrackingMode = .Seated;) () {args.NewSensor.DepthStream.Range = .Default; args.New SkeletonStream.EnableTrackingInNearRange =;}} () {error =;}}}

    The above code can handle the situation where the old Kinect sensor is unplugged, the new sensor is connected, and the depth and bone data streams are turned on. In development, we may connect multiple sensors at the same time. In the try code, we tried to turn on the close-up mode. If the Xbox sensor adopted by the user is not supported, then go to the normal mode. Under normal circumstances, enabling the close-range mode is convenient for debugging and close-range use by the user.

Two Kinect interactive controls
Kinect Region control

    To use Kinect Interaction, you need to add a KinectRegion container class to the interface. KinectRegion is a key element in WPF that uses Kinect Interaction for interaction. It is a container for other Kinect interaction controls. KinectRegion is also responsible for displaying and moving hand icons, that is, the display of the user's hand bone points on the interface. There can be multiple KinectRegions on the main interface of the application, but each KinectRegion cannot be nested, and each KinectRegion can have its own Kinect sensor object.

    Now, add a KinectRegion object to the main form:

    Then, set the KinectSensor property of the kinectRegion to the kinectSensor we obtained:

(! error)
    kinectRegion.KinectSensor = args.NewSensor;
    If you run the program, you will get an error:


    KinectUserViewer shows the depth image data of people in Kinect. If you can't see the image, it means that Kinect is not tracking you at this time, you should try to move to make Kinect track you. Run the program, or use the data recorded by Kinect Studio, then you can see the depth image of your person in the Kinect field of view.


    Then write some hints in the click event:

ButtonOnClick (sender, e)
{
    .Show ();
}
    When our hand does not touch the control, it defaults to blue. When the hand moves over the button, the button becomes larger and a shadow appears around the hand icon to show that it is currently hovering. Then we push the palm in the direction of Kinect, and we will find that the hand icon will animate according to the distance from Kinect, indicating that we are generating the action of pressing the button. When a certain depth threshold is reached, it means that the user has pressed the button, the hand icon will change color, and the Click event will be triggered.


    Repeat the above action, you can see similar animation effects:


    This control is like the standard ScrollViewer in WPF. There is a StackPanell inside to place the scrolling content. The combination of these two controls can make the content scroll horizontally. Now put some KinectCircleButtons in the StackPanel.

(i = 1; i <20; i ++)
{
    button = {
        Content = i,
        Height = 200
    };

    i1 = i;
    button.Click + =
        (o, args) => .Show (+ i1);

    scrollContent.Children.Add (button);
}
    Now run the program, as shown in the animation, at the beginning, you can see the KinectCircleButton below; when the hand moves to the scroll area, the color changes; then close the finger, fist, the gesture icon will change, indicating that you are Do a grab operation on KinectScrollViewer, move your fist, KinectScrollViewer control will paste the gesture icon to the control, just like our fingers operate on the touch screen, and then the button inside will follow the movement of the gesture. When you open your fist, you stop moving. At this time, if you press your hand forward against the desired button, the Click event will be triggered like the previous control.

 Three conclusions

    This article briefly introduces the Kinect Interactions function and related controls introduced in the Kinect for Windows 1.7 SDK. These controls with the Kinect property greatly simplify the difficulty of adding Kinect functionality to our application, allowing us to use Kinect faster and more Okay to create cool Kinect applications. Click here to download the code of this article. Due to the size relationship, there are no two dlls, KinectInteraction170_32.dll or KinectInteraction170_64.dll, you can install the SDK and copy it to the corresponding directory to run. Kinect Interactions-related controls in.