Unity creates Kinect motion detection real-time monitoring waveform graph

Recently, the feature of Kinect posture recognition based on machine learning has been completed. However, the light does a corresponding posture, collect the identified data, always feel passable, less something. And sometimes because of the distance from the Kinect of the situation, the scene interface nothing, had to rely on the feeling to adjust the position. There is, in the face of hundreds of data per second results, it really does not look. So I came up with the idea of whether the collected data could be presented as a real-time monitoring waveform, so that I could handle posture detection data more easily on unity.
OK, let's take a look at the renderings ~

Well, next, let's talk about the specific production process.
The first is the creation of this waveform, which we use in the form of dynamic textures. The
creates a new script, which I name here as Histogramtexture.
Oh, yes, before writing this script, add the Lmnry/setproperty (https://github.com/LMNRY/SetProperty/tree/master/Scripts) plugin to your project first, He can help us when debugging in edit mode, randomly concept the properties of the script and display it.
The following is the code for the main program texture.
We can prefix the class name with a flag [Executeineditmode], which indicates that our code is available in edit mode. The update function in edit mode is not always invoked, but is executed when the scene interface changes.
Remember, in this case, this pattern can be used as the initial painting program texture debugging, when you really run the program, you want to comment out [Executeineditmode].

    Public Material Material = null;
    public bool Iscanvas = FALSE;
    Private texture2d m_generatedtexture = null;
    private int base_width;
    Private Queue heights = new Queue ();
        Private float height = 0f;
    #region Material Properties [Serializefield,setproperty ("Texturewidth")] private int m_texturewidth = 100;
        public int texturewidth{get{return m_texturewidth;
            } set{m_texturewidth = value;
        _updatematerial ();
    }} [Serializefield,setproperty ("Rectanglecolor")] private Color m_rectangalecolor = Color.yellow;
        Public Color rectanglecolor{get{return m_rectangalecolor;
            } set{M_rectangalecolor = value;
        _updatematerial ();
    }} [Serializefield,setproperty ("Count")] private int m_count = 30;
        public int count{get{return m_count; } set{M_count = value;
        _updatematerial (); }} #endregion

The main thing here is to declare the variables required by the script. One of the Serializefield is that this private variable can be exposed to inspector. That SetProperty is the one that I just wanted everyone to download. The token provided by the widget.

    void Start () {
        if (Iscanvas) {

        } else {
            if (material = = null) {
                Renderer Renderer = Gameobject.getcompone Nt<renderer> ();
                if (renderer = = null) {
                    debug.logwarning ("Cannot find a renderer.");
                    return;
                }
                material = renderer.sharedmaterial;
            }
        }

        _updatematerial ();
    }

This code is a bit of a judgment, whether the program texture is used on a 3-dimensional object, or is spent on Unity's UI canvas, and the judge if it is a 3D object, then the object corresponding to the material is not provided.

//Update is called once per frame void _updatematerial () {if (material! = NULL || Iscanvas) {base_width = (int) Mathf.ceil (Texturewidth/(float) m_count);//Debug.Log ("Base_with is
            "+base_width); Heights.
            Clear (); for (int i = 0; i < M_count; i++) {Heights.
            Enqueue (1);
            } m_generatedtexture = _generateproducetexture (); if (Iscanvas) {gameobject.find ("Diagram"). 
            Getcomponent<rawimage> (). texture = M_generatedtexture; } else {material.
            SetTexture ("_maintex", m_generatedtexture); }
        }
    }

Here we call the function that constructs the texture, and do some basic parameter assignment, and here I decide to divide the width of my waveform graph into 30 units. Of course, you can also set a number of other copies. Here I choose a queue as the result of storing each action detection, because the queue is FIFO, so, so that we draw a waveform diagram has a visual effect forward. Because the queue class provided by C # does not give an initialized value, I first put 30 values into the queue, which is the equivalent of initializing. It is also important to note that because the width of each waveform is based on the size of the image and the number of copies generated, so, this value is likely to be a fraction, here, I need to give him to do upward rounding, why is the rounding up. Let's take a look at the following code.

    Private Texture2d _generateproducetexture () {
        texture2d proceduraltexture = new Texture2d (Texturewidth, Texturewidth);
        Draw histogram
        object[] hs = heights. ToArray ();
        for (int w = 0; w < texturewidth; w++) {
            int it = (int) mathf.floor (w/base_width);
            int distogram = Int. Parse (HS [It]. ToString ());          Debug.Log ("Distogram is" +distogram);
            for (int h = 0; h < Distogram; h++) {
                Color pixel = m_rectangalecolor;
                Proceduraltexture.setpixel (W,h,pixel);
            }
        }
        Proceduraltexture.apply ();
        return proceduraltexture;
    }

This code is mainly based on the test results provided by the queue to draw our waveform graph, is to traverse all the map pixel points to assign value, that is, for the current position of the horizontal axis to draw the vertical axis of the corresponding ordinate is how much, and this ordinate is stored in our queue. Our task is to give the position of a horizontal axis, and we can figure out how high the height of this bar is. My calculation method is this kind of int it = (int) mathf.floor (w/base_width); int distogram = Int. Parse (HS [It]. ToString ()); So, if Base_width is rounded down, it will result in it> copies, array out of bounds.

 public void SetHeight (float value) {Maxscore = value > Maxscore? value:maxscore;
        Height = (int) (VALUE*100); Heights.
        Dequeue (); Heights.
        Enqueue (height);
        M_generatedtexture = _generateproducetexture ();
        Debug.Log ("Maxscore is" +maxscore); if (Iscanvas) {gameobject.find ("Diagram"). 
        Getcomponent<rawimage> (). texture = M_generatedtexture; } else {material.
        SetTexture ("_maintex", m_generatedtexture);
    } height = 0f; }

Here is the process of processing the data from the detector and doing the drawing.
about how to set up a detector, you can refer to my other blog post: "Unity5 uses Kinect Studio and gesture Builder to build a custom posture classifier" (http://blog.csdn.net/nijiayy/ article/details/68926979)
If you choose to display the waveform on a plane, remember to have your plane face in direct light, because our material knowledge is the default shader, so does not self-glow, If your plane back to the light, then no matter how the adjustment plane is a piece of black ...
If you choose to display the canvas, select the raw Image object to place under the canvas.
At this point, if this waveform is finished, let's show where Kinect detects our bones.
First, our hierarchy is this:

We need to create a new layer called "Kinectbody"
Canvas : Objects containing diagram and Bodyview two rawimage types
bodymanager : Empty objects, including function Bodysourcemnaager (Kinect library functions)
gesturedetector : Empty objects, including functions Gesturesourcemanager and Gesturecontroller (specifically, the posts I mentioned above)
Creatediagram : Empty object, containing Histogramtexture function
bodyview : Empty object, Contains the Bodyoncanvas function (based on bodysourceview changes)
Detecbodycamera : The camera used to specifically detect the Kinect skeleton, see only the kinectbody layer, The depth is lower than the main camera.
BackGround : equivalent to the background of the skeleton diagram, Kinectbody layer

We need to make a small change based on Bodysourceview's Kinect class library, which is to move his position, and the default Bodysourceview puts our Kinect skeleton in world coordinates (0,0,0), so, Sometimes we need to move it. Also, all the created bones need to be placed in the "kinectbody" layer. Avoid affecting the main camera.

In Bodyoncanvas, the Detecbodycamera is used as the location to provide the display.
Create a render Texture and drag it into the Detecbodycamera as shown in the figure:

In this way, the content of this diagram material is the scene captured by Detecbodycamera and then dragged into the texture parameter of the Bodyview raw Image component.
Adjust the camera location, satisfaction is good ~

Since the Histogramtexture script is actually multi-purpose and can be used both to render the canvas UI and to change the material of the plane in the 3-D world, we provide a iscanvas parameter to judge, but for the use of canvas display, Obviously, we don't need to provide a material to the script, but the inspector of the script is now anyway public and the parameters that need to be serialized are all displayed:

So, to think can not be when the iscanvas tick, the material parameter is not exposed, and when Iscanvas is not checked, material exposed, to this:

This also avoids the use of the script when the miscalculation ~
Since everyone says unity is a self-editing editor, let's say how to edit inspector's display.
Since, add setproperty this plugin, already has the editor this folder, (this control editor's script does not necessarily have to be placed under the Editor folder, but in order to clear the structure of the file, I suggest you put it under the Editor folder) then we'll build a histogramtexture corresponding custom settings panel under this folder.

using unityeditor; using Unityengine; using System; [Customeditor (typeof (Histogramtexture))] public class Histogramtextureeditor:editor {override public void Oninspec
        Torgui () {var myscript = target as histogramtexture;
        Myscript.iscanvas = Editorguilayout.toggle ("Is Canvas", Myscript.iscanvas); if (!myscript.iscanvas) {myscript.material = (material) Editorguilayout.objectfield ("Material", myscript.materi
        Al,typeof (Material));
        } myscript.texturewidth = Editorguilayout.intfield ("Texture Width", myscript.texturewidth);
        Myscript.rectanglecolor = Editorguilayout.colorfield ("Rectangle Color", Myscript.rectanglecolor); MyScript. Count = Editorguilayout.intfield ("Count", MyScript.
    Count); }
}

Note that this script must be named "script name" +editor, which is the script you intend to customize the Inspector panel.
This script is relatively simple, just say something about material, because you find that there is a corresponding field for the parameters of type float, int, and material, texture,gameobject. The answer is Quan ' bu ' y ' og all use all with all with all with all with all with all with all with all with objectfiled, and then add a type of options in it, for example I need to fill in a material, then write:
Myscript.material = (material) Editorguilayout.objectfield ("Material", myscript.material,typeof (material));
OK, so your function of the Inspector panel also seems very interesting ~
If you are interested in this small example, you can download play a Play ~
Https://github.com/ArlexDu/UnityKinectMLTest