Kinect for Windows SDK v2.0 Development Note (13) HD Face Frame (4) Facial Model Builder



(reproduced please specify the source)

Using the SDK: Kinect for Windows SDK v2.0 public preview1409

As before, because the SDK is not completed, no hyperlinks are attached to the function/method/interface.

Let's make facial capture more stable/accurate this time!

Since August with the example of HD face frame, I feel that ifacemodel::getfaceshapedeformations it does not work, has been returning 0.0f of data.

The simple communication with other developers reveals the root of the problem and requires a facial modelbuilder (ifacemodelbuilder).

Well, then this is the beginning. Do you remember the createfacemodel function in the last section? This function needs to pass an array, called

faceshapedeformations Array, here is a simple translation into a polygon bar. The simulated data that stores the feature points of the face, the possible data

For example, the height of the nose simulation value, the detailed need for the official SDK update, and now nothing.

Then, if you have stored the polygon information, you can import it directly whenCreatefacemodel . There is no need to capture and build.

So use Ifacemodelbuilder !

Ihighdefinitionfaceframesource::openmodelbuilder Open the face model builder

Parameter 1: Build properties:

Enum _facemodelbuilderattributes    {        facemodelbuilderattributes_none= 0,        facemodelbuilderattributes _skincolor= 0x1,        facemodelbuilderattributes_haircolor= 0x2    };

Relatively simple, need skin color that is 0x1, need hair color on 0x2, all need to do bitwise OR operation. We only need the polygon data information here,

Facemodelbuilderattributes_none can be.

By the way, we need to support simple data import this time, so this might be the case:

<span style= "FONT-SIZE:14PX;" >    //Create HD facial frame source    if (SUCCEEDED (HR)) {        hr = Createhighdefinitionfaceframesource (M_pkinect, &m_ Phdfaceframesource);    }    Try to read data from a file before creating a facial ModelBuilder    if (SUCCEEDED (HR) &&!read_fsdfile_data ()) {        hr = m_phdfaceframesource-> Openmodelbuilder (Facemodelbuilderattributes_none, &m_pfacemodelbuilder);    }    Start Data collection    if (SUCCEEDED (HR) && m_pfacemodelbuilder) {        hr = m_pfacemodelbuilder-> Beginfacedatacollection ();    } </span>

Ifacemodelbuilder::beginfacedatacollection

Start Data collection

Also easy to understand, the Read_fsdfile_data () method often reads data.

This is how I realized, I am still pure C io better

<span style= "FONT-SIZE:14PX;" >//often try FSD read file bool Thisapp::read_fsdfile_data () {    file* file = nullptr;;    size_t OK = reinterpret_cast<size_t> (m_pfsdfilename);    Open File    if (OK) {        file = _wfopen (M_pfsdfilename, L "RB");        size_t OK = reinterpret_cast<size_t> (file);    }    Read Information    if (OK) {        Auto length = Fread (m_ImagaRenderer.data.sd, 1, Lengthof (M_IMAGARENDERER.DATA.SD), file);        Ok = length = = Lengthof (M_IMAGARENDERER.DATA.SD);    }    Set acquired    if (OK) {        m_bproduced = TRUE;        M_ImagaRenderer.data.co_status = Facemodelbuildercollectionstatus_complete;    }    Close the file    if (file) {        fclose (file), #ifdef _debug        file = nullptr; #endif    }    return ok! = 0;} </span>

Ifacemodelbuilder::get_collectionstatus

Get collection status, collect status has

<span style= "FONT-SIZE:14PX;" >enum _facemodelbuildercollectionstatus    {        facemodelbuildercollectionstatus_complete= 0,        Facemodelbuildercollectionstatus_moreframesneeded= 0x1,        facemodelbuildercollectionstatus_ Frontviewframesneeded= 0x2,        facemodelbuildercollectionstatus_leftviewsneeded= 0x4,        Facemodelbuildercollectionstatus_rightviewsneeded= 0x8,        facemodelbuildercollectionstatus_ tiltedupviewsneeded= 0x10    };</span>

Facemodelbuildercollectionstatus_complete

The facial information is collected, this is the state.

facemodelbuildercollectionstatus_moreframesneeded

Additional location information is required, and the next 4 states exist in any state, which is set to 1

facemodelbuildercollectionstatus_frontviewframesneeded

Front frame data is required and the face is facing the Kinect

facemodelbuildercollectionstatus_leftviewsneeded

Need to face to the left (almost 90 degrees)

facemodelbuildercollectionstatus_rightviewsneeded

Requires a face to the right (almost 90 degrees)

facemodelbuildercollectionstatus_tiltedupviewsneeded

It needs to be tilted upwards (45 degrees for visual inspection), but the author's Kinect is placed high, which requires a 90-degree head

It is also worth mentioning that headphones are best removed and will increase the probability of successful collection.

typos do not care, the code has been uploaded, change is meaningless.

Ifacemodelbuilder::get_capturestatus

Get the acquisition state, is to prompt the user's posture is how wrong, basic does not move is OK

Enum _facemodelbuildercapturestatus    {        facemodelbuildercapturestatus_goodframecapture= 0,        facemodelbuildercapturestatus_otherviewsneeded= 1,        facemodelbuildercapturestatus_lostfacetrack= 2,        Facemodelbuildercapturestatus_facetoofar= 3,        facemodelbuildercapturestatus_facetoonear= 4,        facemodelbuildercapturestatus_movingtoofast= 5,        facemodelbuildercapturestatus_systemerror= 6    };

It's not much.

After the acquisition is successful, the ifacemodelbuilder::getfacedata can be used to obtain ifacemodeldata,

actually not lowercase, what Microsoft is thinking ╮ (￣▽￣) ╭

After getting ifacemodeldata::P roducefacemodel to create a facial model, remember to release the previous model yo

The code is almost like this, please note that I found the build ifacemodeldata after the experiment ::P Roducefacemodel model takes a few seconds,

It simply affects the user experience, so it's recommended to keep the data, next

    Check the face ModelBuilder if (SUCCEEDED (HR) &&!m_bproduced) {ifacemodeldata* pfacemodeldata = nullptr;        Check the collection status hr = M_pfacemodelbuilder->get_collectionstatus (&m_imagarenderer.data.co_status); Check the acquisition status if (SUCCEEDED (HR)) {hr = M_pfacemodelbuilder->get_capturestatus (&m_imagarenderer.data.c        A_status); }//Capture successful acquisition of data if (SUCCEEDED (hr) && M_ImagaRenderer.data.co_status = = Facemodelbuildercollectionstat        Us_complete) {hr = M_pfacemodelbuilder->getfacedata (&pfacemodeldata);            }//Generate facial model if (SUCCEEDED (HR) && pfacemodeldata) {saferelease (M_pfacemodel);        hr = Pfacemodeldata->producefacemodel (&m_pfacemodel);            }//Check results if (SUCCEEDED (HR) && pfacemodeldata) {m_bproduced = TRUE; By the way output data m_pfacemodel->getfaceshapedeformations (Lengthof (M_IMAGARENDERER.DATA.SD), M_imagarendeRER.DATA.SD);    }//Release off Saferelease (Pfacemodeldata); }

It's almost like this:

Let's talk about the collected information:

The author's approach is natural and simple:

          Case Wm_keydown:if (WParam = = ' S ') {file* FILE = nullptr;                    BYTE Buffer[faceshapedeformations_count * 64];                    size_t now_length = 0; if (!_wfopen_s (&file, L "FaceShapeDeformations.txt", L "WB")) {memcpy (buffer, Pourapp->m_imaga                        RENDERER.DATA.SD, sizeof (POURAPP-&GT;M_IMAGARENDERER.DATA.SD));                        char* index = reinterpret_cast<char*> (buffer+sizeof (POURAPP-&GT;M_IMAGARENDERER.DATA.SD)); *index = ' \ r '; ++index; *index = ' \ n ';                        ++index;                        float* Data_index = pourapp->m_imagarenderer.data.sd;  for (int i = 0; i < Faceshapedeformations_count; ++i) {now_length = sprintf (Index, "%f\r\n",                            *data_index);                            Index + = now_length;                        ++data_index; } fwrite (buffer, 1, reinterpret_cast<byte*> (index)-buffer, file);                        } if (file) {fclose (file);                        File = nullptr;                    :: Sendmessagew (hwnd, WM_CLOSE, 0, 0);                    } else{:: MessageBoxW (hwnd, L "Save failed", L "Error", mb_iconerror); }} break;

But the data for the simple two samples are as follows:

It can be seen that the data fluctuation is large, the solution is only multiple sampling offset. Algorithm many, the simplest averaging method.

It can also be assumed that the normal distribution of n (μ,σ^2), using the method of "probability theory and Mathematical Statistics", does not say much (in fact, it is not).

Debugging can pass a parameter that represents the path to the file:

The formal use of more simple, directly drag the output file to the exe file.

Example: click here

This time I learned, early upload, but typos



Kinect for Windows SDK v2.0 Development Note (13) HD Face Frame (4) Facial Model Builder