Introduction to the Kinect for Windows SDK Development (ix) skeleton tracking advanced

1. User interaction based on depth of data

So far we have only used the X,y value of the joint point in the skeleton data. However, the joint point data produced by Kinect has a depth value in addition to the X,y value. A Kinect based application should take advantage of this depth value. The following sections describe how to use depth values in the Kinect application.

In addition to using the 3D features of WPF, the dimensions of the visual elements can be set in the layout system according to the depth value to achieve the stereo effect of some program. The following example uses the Canvas.ZIndex property to set the level of the element, manually set the size of the control, and use ScaleTransform to scale according to the change in the depth value. The user interface includes some circles, and each circle represents a certain depth. The application tracks the user's hand joints and displays them as hand-shaped icons, which are scaled according to the depth values of the user's hand joints, the closer the user is to the Kinect, the larger the hand chart, the smaller the contrast.

To create a new WPF project, the XAML for the main interface is as follows. The main layout container is the Cnavas container. It contains 5 ellipses and corresponding TextBlock controls, TextBlock used to describe the circle. These circles are randomly distributed on the screen, but the canvas.zindex of the circle is OK. The canvas container also contains two image controls to represent two hands. Each hand icon defines a ScaleTransform object. Hand-shaped icons are aligned with the right hand direction, and the ScaleTransform ScaleX set to 1 can be reversed, looking like the left hand.

<window x:class= "Kinectdepthbasedinteraction.mainwindow" xmlns= "http://schemas.microsoft.com/winfx/2006/xaml/  Presentation "xmlns:x=" Http://schemas.microsoft.com/winfx/2006/xaml "title=" Depth UI Target "height=" 1080 " Width= "1920" windowstate= "maximized" background= "white" > <Window.Resources> <style x:key= "Targetl Abel "targettype=" TextBlock "> <setter property=" fontsize value= "/>" <setter  rty= "foreground" value= "white"/> <setter property= "fontweight" value= "Bold"/>
            
        Property= "ishittestvisible" value= "False"/> </Style> </Window.Resources> <Viewbox> <grid x:name= "LayoutRoot" width= "1920" height= "1280" > <image x:name= "Depthimage" /> <stackpanel horizontalalignment= "left" verticalalignment= "top" > <textblock X:na Me= "Debuglefthand" style="{StaticResource Targetlabel}" foreground= "Black"/> <textblock x:name= "Debugrighthand" Style= "{Stat
                Icresource Targetlabel} "foreground=" Black "/> </StackPanel> <Canvas> <ellipse x:name= "Target3" fill= "Orange" height= "" width= "", canvas.left= "776" canvas.top= "162" canvas.zindex= "1040"/> <textblock text= "3" canvas.left= "860" canvas.top= "206" panel.zindex= "1040" style= "{StaticR Esource Targetlabel} "/> <ellipse x:name=" Target4 "fill=" Purple "height=" width= "" s.left= "732" canvas.top= "canvas.zindex=" 940 "/> <textblock text=" 4 "canvas.left=" 840 "Canvas.To p= "372" panel.zindex= "940" style= "{StaticResource Targetlabel}"/> <ellipse x:name= "Target5" Fi Ll= "Green" height= "width=" canvas.left= "880" canvas.top= "592" canvas.zindex= "840"/> <textb Lock text= "5" canvas.left="908" canvas.top= "590" panel.zindex= "840" style= "{StaticResource Targetlabel}"/> <ellipse x:Na
                Me= "Target6" fill= "Blue" height= "width=" canvas.left= "352" canvas.top= "544" canvas.zindex= "740"/> <textblock text= "6" canvas.left= "368" canvas.top= "582" panel.zindex= "740" style= "{StaticResource TargetLabel}"/&G
    
                T <ellipse x:name= "Target7" fill= "Red" height= "" width= "" "canvas.left=" "378," canvas.top= "" "canvas.zindex=" 640 " /> <textblock text= "7" canvas.left= "422" "canvas.top=" "panel.zindex=" 640 "style=" {StaticResource Targetlabel} "/> <image x:name=" lefthandelement "source=" Images/hand.png "width=" "Height=" 80 "rendertransformorigin=" 0.5,0.5 "> <Image.RenderTransform> <scalet
    Ransform x:name= "Lefthandscaletransform" scalex= "1" centery= "-1"/> </Image.RenderTransform>            </Image> <image x:name= "righthandelement" source= "images/hand.pn
                        G "Width=" "height=" rendertransformorigin= "0.5,0.5" > <Image.RenderTransform> <scaletransform x:name= "Righthandscaletransform" centery= "1" scalex= "1"/> </im Age.
rendertransform> </Image> </Canvas> </Grid> </Viewbox> </window

A circle of different colors represents a different depth, such as an element named Target3, representing a distance of 3 feet. The Target3 is larger than the TARGET7, which can be achieved simply by scaling. In our instance program, we hard-code its size, and in actual programs, it should be scaled according to specific requirements. The canvas container arranges the elements in the direction perpendicular to the computer screen based on the Canvas.ZIndex values of the child elements, such as the topmost element, whose canvas.zindex is the largest. If two elements have the same zindex value, they are displayed according to the order in which they are declared in XAML, and in XAML, the elements that are later declared precede the previously declared elements. For all child elements of a canvas, the larger the ZIndex value, the closer the screen is, the smaller the distance from the screen. The depth value is reversed just to achieve the desired effect. This means that we cannot use the depth value directly to assign a value to the zindex, but to do a bit of conversion. The maximum depth that Kinect can produce is 13.4 feet, and correspondingly, we set the range of Canvas.ZIndex to 0-1340, multiplying the depth value by 100 to get better accuracy. Therefore, the Target5 Canvas.ZIndex is set to 840 (13.5-5=8.4*100=840).