Kinect for Windows SDK v2.0 Development Notes (v) skeleton frame with smiley man

(reprint please indicate the source)

Using the SDK: Kinect for Windows SDK v2.0 Public Preview

This time to talk about the acquisition of skeleton frames. Well, that's what Kinect bought for it. Otherwise, you can buy other products, and the Kinect selling point is this.

Let's take a look at this support bone joint:

Enum _jointtype
    {
        Jointtype_spinebase	= 0,
        jointtype_spinemid	= 1,
        jointtype_neck	= 2,
        Jointtype_head	= 3,
        jointtype_shoulderleft	= 4,
        jointtype_elbowleft	= 5,
        Jointtype_wristleft	= 6,
        jointtype_handleft	= 7,
        jointtype_shoulderright	= 8,
        Jointtype_elbowright	= 9,
        jointtype_wristright	= ten,
        jointtype_handright	= one,
        Jointtype_hipleft	= Jointtype_kneeleft =
        ,
        jointtype_ankleleft
        =, Jointtype_footleft	=
        jointtype_hipright = m,
        jointtype_kneeright	=
        MB, Jointtype_ankleright	= Jointtype_footright =
        ,
        jointtype_spineshoulder	=
        Jointtype_handtipleft	= Jointtype_thumbleft =
        ,
        jointtype_handtipright
        = Jointtype_thumbright	=
        Jointtype_count	= (jointtype_thumbright + 1) 
    };

Support these 25 joints, do not rule out the possibility of increase, after all, close-up can be distinguished by ten fingers.

The state of each joint is described in this structure:

typedef struct _JOINT
    {
    jointtype jointtype;
    Cameraspacepoint Position;
    Trackingstate trackingstate;
    } 	Joint;

Jointtype is the previous joint number, position is the Kinect's camera space coordinates and is three-dimensional. Trackingstate is the current tracking state of the joint,

There are: No Trace (0), position is speculated (1), Position is traced (2)

It is worth noting that the C + + SDK also provides a judgment on the status of the hand:

Enum _handstate
    {
        Handstate_unknown	= 0,
        handstate_nottracked	= 1,
        handstate_open	= 2,
        handstate_closed	= 3,
        handstate_lasso	= 4
    };

There are: Unknown (0), not tracked (1), Spread out (2), fist (3) and Lasso (4), lasso do not know how to translate, is probably in the state between the spread and the fist,

For example: Or even this can be called lasso.

Use the same method as before, say different:

        ibody* Ppbodies[body_count] = {0};

        if (SUCCEEDED (HR))
        {
            hr = Pbodyframe->getandrefreshbodydata (Body_count, ppbodies);
        }

So get 6 Ibody interface, run out remember to release, a loop released

Each interface uses code similar to the following to get the data:

       for (int i = 0; i < Nbodycount ++i)
       {
           ibody* pbody = ppbodies[i];
           if (pbody)
           {
               BOOLEAN btracked = false;
               hr = pbody->get_istracked (&btracked);

               if (SUCCEEDED (HR) && btracked)
               {
                   joint joints[jointtype_count];
                   Handstate lefthandstate = Handstate_unknown;
                   Handstate righthandstate = Handstate_unknown;

                   Pbody->get_handleftstate (&lefthandstate);
                   Pbody->get_handrightstate (&righthandstate);

                   hr = pbody->getjoints (_countof (joints), joints);
                   if (SUCCEEDED (HR))
                   {
                       //XXXXXXXX
                   }
               }
           }

The code is simple and can be seen from the name of the method. The visualization method here is to use Microsoft to provide the method in the SDK example,

Probably is connected to the joint use straight line up and so on, this part of the code is quite boring, there are direct2d students can ignore, please see the example details.
The effect is as follows

Well, in fact, the previous example SDK provides a number of examples, here naturally to give an original thing (of course, creativity is not original).

See the title of the classmate probably can guess, that is, "attack Shell Mobile Team S.A.C" (Ghost in the Shell:stand Alone Complex) inside a character, the character out of use an icon to block the face:

Here we are going to achieve this effect, is a real-time coding software bar.

Here, our text will also be like the original rotation, in order to ensure the flow, so our image API selected D2D 1.1 (Can wait in the vertical sync).

D2D 1.1 initialization, to tell the truth, I don't remember,

To use D2D 1.1, copy over, after all, not a test

So how do you implement that icon, but you use the picture. But as a program ape, it's a good choice to use code for instant generation.

DIRECT2D provides a hardware-accelerated geometry rendering interface that is very convenient. Rotated text rendering requires Directwrite + DIRECT2D, the range of learning is not here,

Please see examples in detail.

As for the geometrical shape of this icon, of course, with visual ... It's not realistic that I found a smiley man in the other place SVG image, the code is as follows:

<?xml version= "1.0" encoding= "Utf-8"?> <svg xmlns= "http://www.w3.org/2000/svg" xmlns:xlink= "http://" Www.w3.org/1999/xlink "viewbox=" -160-160 360, "> <path id=" F "d=" m123,0a123,123 0,0 1-246,0a123,123 0,0 1 246 , 0 "/> <g fill=" #057 "> <circle r=" 160 "/> <circle r=" i "fill=" #fff "/> <text F Ont-size= "font-stretch=" "Condensed" font-family= "Impact" > <animatetransform type= "Rotate" from= "360 0 0" to  = "0 0 0" dur= "10s" attributename= "transform" repeatcount= "indefinite"/> <textpath xlink:href= "#f" >i thought What I ' d do is, I ' d pretend I was one of those deaf-mutes</textpath> </text> <circle r=  
    T <circle r= "fill=" "#fff"/> <path d= "m-8-119h16 l2,5h-20z"/> <circle "cx= 160" cy= "0" r= ""/&G  
    T <path d= "m-95-20v-20h255a40,40 0,0 1 0,80h-55v-20z"/> <path d= "m-85 0a85,85 0,0 0 170,0h-20a65,65 0,0 1-130 , 0z "/>;p ath d= "m-65 20v20h140v-20z"/> <path d= "m-115-20v10h25v30h250a20,20 0,0 0 0,-40z" fill= "#fff"/> <  
  Path d= "m-20 10c-17-14-27-14-44 0 6-25 37-25 0z"/> <path d= "M60 10c-17-14-27-14-44 0 6-25 37-25 0z"/>
 </g> </svg>

Now according to this code, you can probably write the following code, part of the coordinates after my fine-tuning.

Create smiley male related HRESULT Imagerenderer::createlaughingman () {//Basic radius const FLOAT Base_radius = 135.F;
    HRESULT hr = S_OK;
    idwritetextformat* Pimpactformat = nullptr; Create Impact text Format hr = M_pdwritefactory->createtextformat (L "Impact", nullptr, Dwrite_font_we 
        Ight_normal, Dwrite_font_style_normal, dwrite_font_stretch_condensed, 41.f/96.f*72.f, L "",

    &pimpactformat);
        Create a text layout if (SUCCEEDED (HR)) {pimpactformat->setwordwrapping (dwrite_word_wrapping_no_wrap);
        wchar* Text = L "I thought what I ' d do is, I ' d pretend I was one of those deaf-mutes";
        Auto length = wcslen (text); hr = m_pdwritefactory->createtextlayout (text, length, Pimpactformat, Base_radius, Base_radius, &m_
    Ptextlayoutlaughingman);
        //Create Text geometry path: A round if (SUCCEEDED (HR)) {D2d1_ellipse ELLIPSE;
        Ellipse.point.x = 0.F;
        Ellipse.point.y = 0.F; EllipSe.radiusx = Base_radius;
        Ellipse.radiusy = Base_radius;
    hr = M_pd2dfactory->createellipsegeometry (&ellipse, &m_ptextanimationpath); 
        }//Smiley male path if (SUCCEEDED (HR)) {hr = M_pd2dfactory->createpathgeometry (&m_plaughingmangeometryblue);
        Draw line id2d1geometrysink* Psink = nullptr;
        if (SUCCEEDED (HR)) {hr = M_plaughingmangeometryblue->open (&psink);
            } if (SUCCEEDED (HR)) {Auto Nowpoint = d2d1::P oint2f ();
            Psink->setfillmode (d2d1_fill_mode_winding);
            D2d1_arc_segment ARC;
            D2d1_bezier_segment Bézier;
            Arc.rotationangle = 0.F; <path d= "m-8-119h16 l2,5h-20z"/> nowpoint.x = -8.f;
            Nowpoint.y = -124.F;
            Psink->beginfigure (Nowpoint, d2d1_figure_begin_filled);
            Nowpoint.x + 16.f;
            Psink->addline (Nowpoint); Nowpoint.x + 2.f;
        Nowpoint.y + 5.f;    Psink->addline (Nowpoint);
            Nowpoint.x = 20.F;
            Psink->addline (Nowpoint);
            Psink->endfigure (d2d1_figure_end_closed); <path d = "m-95-20v-20h255a40,40 0,0 1 0,80h-55v-20z"/> Nowpoint.x = -105.F;
            Nowpoint.y = -20.F;
            Psink->beginfigure (Nowpoint, d2d1_figure_begin_filled);
            Nowpoint.y = 20.F;
            Psink->addline (Nowpoint);
            Nowpoint.x + 270.f;
            Psink->addline (Nowpoint);
            Nowpoint.y + 80.f;
            Arc.size.height = 40.F;
            Arc.size.width = 40.F;
            Arc.sweepdirection = d2d1_sweep_direction_clockwise;
            Arc.point = Nowpoint;
            Arc.arcsize = D2d1_arc_size_small;
            Psink->addarc (&AMP;ARC);
            Nowpoint.x = 55.F;
            Psink->addline (Nowpoint);
            Nowpoint.y = 20.F;
            Psink->addline (Nowpoint);
            Nowpoint.x + 55.f; Psink-&gT
            AddLine (Nowpoint);
            Nowpoint.y = 40.F;
            Arc.size.height = 20.F;
            Arc.size.width = 20.F;
            Arc.sweepdirection = d2d1_sweep_direction_counter_clockwise;
            Arc.point = Nowpoint;
            Psink->addarc (&AMP;ARC);
            Psink->endfigure (d2d1_figure_end_closed); <path d= "m-85 0a85,85 0,0 0 170,0h-20a65,65 0,0 1-130,0z"/> nowpoint.x = -85.f;
            Nowpoint.y= 20.f;
            Psink->beginfigure (Nowpoint, d2d1_figure_begin_filled);
            Nowpoint.x + 170.f;
            Arc.size.height = 90.F;
            Arc.size.width = 90.F;
            Arc.sweepdirection = d2d1_sweep_direction_counter_clockwise;
            Arc.arcsize = D2d1_arc_size_small;
            Arc.point = Nowpoint;
            Psink->addarc (&AMP;ARC);
            Nowpoint.x = 20.F;
            Psink->addline (Nowpoint);
            Nowpoint.x = 130.F;
            Arc.size.height = 70.F; Arc.size.width = 70.F;
            Arc.sweepdirection = d2d1_sweep_direction_clockwise;
            Arc.point = Nowpoint;
            Psink->addarc (&AMP;ARC);
            Psink->endfigure (d2d1_figure_end_closed); <path d= "m-65 20v20h130v-20z"/> nowpoint.x = -65.f;
            Nowpoint.y = 20.F;
            Psink->beginfigure (Nowpoint, d2d1_figure_begin_filled);
            Nowpoint.y + 20.f;
            Psink->addline (Nowpoint);
            Nowpoint.x + 130.f;
            Psink->addline (Nowpoint);
            Nowpoint.y = 20.F;
            Psink->addline (Nowpoint);
            Psink->endfigure (d2d1_figure_end_closed);
            Psink->beginfigure (Nowpoint, d2d1_figure_begin_filled); <path d = "m-20 10c-17-14-27-14-44 0 6-25 37-25 0z"/> Nowpoint.x = -20.F;
            Nowpoint.y = 10.F;
            Psink->beginfigure (Nowpoint, d2d1_figure_begin_filled);
            bezier.point1.x = NOWPOINT.X-17.F; Bezier.poInt1.y = NOWPOINT.Y-14.F;
            bezier.point2.x = NOWPOINT.X-27.F;
            Bezier.point2.y = NOWPOINT.Y-14.F;
            Nowpoint.x = 44.F;
            Bezier.point3 = Nowpoint;
            Psink->addbezier (&bezier);
            bezier.point1.x = Nowpoint.x + 6.f;
            Bezier.point1.y = NOWPOINT.Y-25.F;
            bezier.point2.x = Nowpoint.x + 37.f;
            Bezier.point2.y = NOWPOINT.Y-25.F;
            Nowpoint.x + 44.f;
            Bezier.point3 = Nowpoint;
            Psink->addbezier (&bezier);
            Psink->endfigure (d2d1_figure_end_closed); <path d = "M60 10c-17-14-27-14-44 0 6-25 37-25 0z"/> Nowpoint.x = 60.F;
            Nowpoint.y = 10.F;
            Psink->beginfigure (Nowpoint, d2d1_figure_begin_filled);
            bezier.point1.x = NOWPOINT.X-17.F;
            Bezier.point1.y = NOWPOINT.Y-14.F;
            bezier.point2.x = NOWPOINT.X-27.F; BEZIER.POINT2.Y = Nowpoint.y-14.F;
            Nowpoint.x = 44.F;
            Bezier.point3 = Nowpoint;
            Psink->addbezier (&bezier);
            bezier.point1.x = Nowpoint.x + 6.f;
            Bezier.point1.y = NOWPOINT.Y-25.F;
            bezier.point2.x = Nowpoint.x + 37.f;
            Bezier.point2.y = NOWPOINT.Y-25.F;
            Nowpoint.x + 44.f;
            Bezier.point3 = Nowpoint;
            Psink->addbezier (&bezier);
            Psink->endfigure (d2d1_figure_end_closed);
        hr = Psink->close ();
    } saferelease (Psink); }//Smiley male White part if (SUCCEEDED (HR)) {hr = M_pd2dfactory->createpathgeometry (&m_plaughingmangeometrywhit
        e);
        Draw line id2d1geometrysink* Psink = nullptr;
        if (SUCCEEDED (HR)) {hr = M_plaughingmangeometrywhite->open (&psink);
            } if (SUCCEEDED (HR)) {Auto Nowpoint = d2d1::P oint2f (); <path d = "m-115-20v10h25v30h250a20,20 0,0 0 0,-40z" Fill = "#fff"/> Nowpoint.x = -125.F;
            Nowpoint.y = -20.F;
            Psink->beginfigure (Nowpoint, d2d1_figure_begin_filled);
            Nowpoint.y + 10.f;
            Psink->addline (Nowpoint);
            Nowpoint.x + 35.f;
            Psink->addline (Nowpoint);
            Nowpoint.y + 30.f;
            Psink->addline (Nowpoint);
            Nowpoint.x + 260.f;
            Psink->addline (Nowpoint);
            Nowpoint.y = 40.F; D2d1_arc_segment ARC = {nowpoint, D2d1::sizef (20.F, 20.F), 0.F, D2d1_sweep_direction_counter_clockwise, D2d1_arc_size_
            SMALL};
            Psink->addarc (&AMP;ARC);
            Psink->endfigure (d2d1_figure_end_closed);
        hr = Psink->close (); }//Smiley Blue if (SUCCEEDED (HR)) {hr = M_pd2ddevicecontext->createsolidcolorbrush (D2d1::colorf 0x00557
    7), &m_plaughingbluebrush); }//Smiley white if (SUCCEEDED (HR)) {hr = M_pd2ddevicecontext->createsolidcolorBrush (D2d1::colorf (0xFFFFFF), &m_plaughingwhitebrush);
    } saferelease (Pimpactformat);
return HR; }

It's a sore egg anyway.

Yes, we are using D2D 1.1 this time. I've said that before, so use polling mode here.

To use color data flow + Skeleton data flow, then whether to use the complex source frame.

The use of complex source frames to ensure data synchronization, polling mode asynchronous effect is timely, so there is no use of complex source frames.

The approximate process is as follows:

Refresh:

Get color data-> copy in place diagram

Get Skeleton data-> Check and update the head position, according to the distance set relative zoom rate (achieve near large far small, roughly can, without precision)

Rendering:

Render Color Frames

Rendering Smiley Man

KINECT2 supports 6 human skeleton tracking, so the data should be prepared in 6 copies. So that 6 people will be able to play code together.

The results are as follows: The skeleton data obtained by KINECT2 has been smoothed out by default, and it does not need to be set smoothing parameters like 1 generation, and also indicates the improvement of precision.

In order to simulate the original jitter, you can only manually simulate.

Example Download Address: Click here