Leap Motion tracker hands, fingers, and tools

Leap Motion translation series Article http://52coding.com/leap-motion-official-doc-translation

Tracker hands, fingers, and tools

Hand, finger, and tool are basic tracing entities of the Leap Motion System. This article will discuss in detail how to obtain and use these object.

Overview

The Leap API defines a class that can represent each object that is basically tracked. The frame object provides a list of accessible hands, fingers, and tools. Tip objects and tools are tip objects, and they can be put into the tip list PointableList at the same time, or use the finger list FingerList and tool list ToolList. Hand objects provide access to their fingers and tools (both in the tip list or separated ). Features of hand, finger, and tool objects are reported under the Leap coordinate system (measured in millimeters. The Leap SDK provides a Vector class to describe points and orientations. Vector classes provide some useful mathematical functions for vector operations. [This is good, so you can avoid writing it yourself].

Hand and tip list

List classes all have similar structures. They are designed as Vector Form matrices and support iterators. You cannot remove or change the member variables in the list received from the Leap API, but you can combine objects of the same type in the list. Use iterator for a list:

for(HandList::const_iterator hl = handList.begin(); hl != handList.end();)                std::cout << *hl << std::endl;

The hand, tip object, finger, and tool List defines additional functions based on the relative position in the Leap coordinate system to obtain member variables in the list. These functions include leftmost (), rightmost (), and frontmost (). The following code snippet illustrates how to use these functions:

Leap::Finger farLeft = frame.fingers().leftmost();Leap::Finger mostForwardOnHand = frame.hands()[0].fingers().frontmost();Leap::Tool rightTool = frame.tools().rightmost();

A more complex example is to calculate a rectangle that contains the boxes of all detected tip objects. Because it is not provided in the API, this example defines its own function to obtain the tip at the top, bottom, and back.

float left = frame.pointables().leftmost().tipPosition().x;    float right = frame.pointables().rightmost().tipPosition().x;    float front = frame.pointables().frontmost().tipPosition().z;    float back = backmost(frame.pointables()).tipPosition().z;    float top = topmost(frame.pointables()).tipPosition().y;    float bottom = bottommost(frame.pointables()).tipPosition().y;    Leap::Pointable backmost(PointableList pointables)    {        if(pointables.count() == 0) return Leap::Pointable::invalid();        Leap::Pointable backmost = pointables[0];        for( int p = 1; p < pointables.count(); p++ )        {            if( pointables[p].tipPosition().z > backmost.tipPosition().z)                backmost = pointables[p];        }        return backmost;    }    Leap::Pointable topmost(PointableList pointables)    {        if(pointables.count() == 0) return Leap::Pointable::invalid();        Leap::Pointable topmost = pointables[0];        for( int p = 1; p < pointables.count(); p++ )        {            if( pointables[p].tipPosition().y > topmost.tipPosition().y)                topmost = pointables[p];        }        return topmost;    }    Leap::Pointable bottommost(PointableList pointables)    {        if(pointables.count() == 0) return Leap::Pointable::invalid();        Leap::Pointable bottommost = pointables[0];        for( int p = 1; p < pointables.count(); p++ )        {            if( pointables[p].tipPosition().y < bottommost.tipPosition().y )                bottommost = pointables[p];        }        return bottommost;    }

[That is, it is easy to understand that the minimum external rectangle of the three-dimensional coordinates of all objects is calculated ~ The code is exactly the same. After reading the API and my own guesses, Leap Motion is quite concerned about the left, front, and right directions, and it may not be able to accurately detect the High-Leap, but why not provide functions directly in other directions?]

Hand

The hand class describes a physical hand that is tracked by the Leap. A hand object provides a list of access to its own tip (finger), as well as a property that describes the coordinates, orientation, and motion of the hand. Get a hand object from a frame:

Leap::Frame frame = controller.frame(); // controller is a Leap::Controller objectLeap::HandList hands = frame.hands();Leap::Hand firstHand = hands[0];

Or, if you know the ID in the previous frame:

Leap::Hand knownHand = frame.hand(handID);

You can also obtain the hand object through their relative positions in the frame:

Leap::Frame frame = controller.frame(); // controller is a Leap::Controller objectLeap::HandList hands = frame.hands();Leap::Hand leftmost = hands.leftmost();Leap::Hand rightmost = hands.rightmost();Leap::Hand frontmost = hands.frontmost();

Note that the rightmost () and rightmost () functions only distinguish the rightmost or leftmost operator. This method does not distinguish whether the hand is the left hand or the right hand.

Obtain hand features

A hand is described by its coordinates, orientation, and motion. Hand coordinates are obtained through the palm coordinate attributes. a vector of three-dimensional coordinates in the palm center is provided, with Leap Motion as the origin and millimeters as the unit. The orientation of the hand is given by two vectors: Direction-pointing from the center of the palm of the hand to the direction of the finger, palm vertical-pointing to the outside of the palm, perpendicular to the plane of the hand. The motion of a hand is given by the motion speed attribute. It is a vector describing the instantaneous motion of a hand in millimeters per second. You can also obtain the motion factor, which can describe the changes in the displacement, rotation, and scaling values of a hand at two frames. The following code block describes how to obtain a hand object and its basic attributes in a frame:

    Leap::Hand hand = frame.hands().rightmost();    Leap::Vector position = hand.palmPosition();    Leap::Vector velocity = hand.palmVelocity();    Leap::Vector direction = hand.direction();

Getting tips and tools

You can get the finger and tool associated with the hand through the list or just using the ID of the previous frame. Using the list method:

    // hand is a Leap::Hand object    Leap::PointableList pointables = hand.pointables(); // Both fingers and tools    Leap::FingerList fingers = hand.fingers();    Leap::ToolList tools = hand.tools();

Use the method of the previous frame ID: Leap: Pointable knownPointable = hand. pointable (pointableID); to obtain the relative position of the finger or tool in the Leap field of view, you can use the right-, left-, and frontmost functions in the matching list class.

    // hand is a Leap::Hand object    Leap::Pointable leftPointable = hand.pointables().leftmost();    Leap::Finger rightFinger = hand.fingers().rightmost();    Leap::Tool frontTool = hand.tools().frontmost();

Note that these functions are relative to the Leap Motion origin, rather than its own. To obtain the finger relative to the hand, you can use the Leap matrix class to convert the finger coordinate to the hand of the reference frame. [Not quite familiar]

Computing hand orientation

Using the hand direction and vertical vector, you can calculate the shot orientation angle (relative to the east ). The vector class defines the Pitch rotation (around the X axis rotation), the left and right Yaw rotation (around the Y axis rotation), and the plane Roll rotation (around the Z axis rotation ):

    float pitch = hand.direction().pitch();    float yaw = hand.direction().yaw();    float roll = hand.palmNormal().roll();

Note that the roll function only provides the pre-judgment angle when vertical vectors are used. [It seems that roll is difficult to judge]

Convert the finger coordinates to the reference frame of the hand (the hand is the reference object)

Sometimes, it is a common method to obtain the finger coordinates on the hand through the reference frame of the hand. This method allows you to sort your fingers in space and analyze the coordinates of your fingers. You can use the Leap matrix class to construct a conversion matrix to convert finger coordinates and direction coordinates. The frame of the hand in the reference can be effectively defined by the orientation of the hand and the vertical vector of the palm (by the third coordinate axis orthogonal to the two coordinate axes. This method causes the X axis to move along the side of the hand, while the Z axis points to the front, and the Y axis is parallel to the palm plane.

    Leap::Frame frame = leap.frame();    for( int h = 0; h < frame.hands().count(); h++ )    {        Leap::Hand leapHand = frame.hands()[h];        Leap::Vector handXBasis =  leapHand.palmNormal().cross(leapHand.direction()).normalized();        Leap::Vector handYBasis = -leapHand.palmNormal();        Leap::Vector handZBasis = -leapHand.direction();        Leap::Vector handOrigin =  leapHand.palmPosition();        Leap::Matrix handTransform = Leap::Matrix(handXBasis, handYBasis, handZBasis, handOrigin);        handTransform = handTransform.rigidInverse();        for( int f = 0; f < leapHand.fingers().count(); f++ )        {            Leap::Finger leapFinger = leapHand.fingers()[f];            Leap::Vector transformedPosition = handTransform.transformPoint(leapFinger.tipPosition());            Leap::Vector transformedDirection = handTransform.transformDirection(leapFinger.direction());            // Do something with the transformed fingers        }    }

 

Tip

Tip objects can be represented as fingers and tools-that is, things that can point out the direction. You can obtain the fingers and tools associated with the hand by using a specific hand in the hand object. The tip does not need to be associated with the hand object-the physical hand may not be in the field of view or be blocked by the other hand. Therefore, the tip list in the frame can contain fingers and tools that are not associated with the hand [This increases applicability and introduces complexity]. Tip objects have many attributes to describe fingers and tools: # tip coordinates-instantaneous coordinates in millimeters in the Leap Motion coordinate system. # Tip speed-instantaneous speed in millimeters per second. # Pointing --- current pointing to the orientation vector. # Length
--- The length of the finger and tool display. # Width --- average width. # Touch distance-the distance after the virtual touch plane is restored. # Touch area-the relationship between the current tip and the virtual touch plane. The following example shows how to obtain short objects and basic access attributes from frames:

    Leap::Pointable pointable = frame.pointables().frontmost();    Leap::Vector direction = pointable.direction();    float length = pointable.length();    float width = pointable.width();    Leap::Vector stabilizedPosition = pointable.stabilizedTipPosition();    Leap::Vector position = pointable.tipPosition();    Leap::Vector speed = pointable.tipVelocity();    float touchDistance = pointable.touchDistance();    Leap::Pointable::Zone zone = pointable.touchZone();

 

Convert a tip object to a finger or tool

To convert a tip object to a proper finger and tool subclass, you need to use the appropriate finger and tool Constructor (sometimes you should use the Leap class constructor) [not quite clear].

 if (pointable.isTool()) {        Leap::Tool tool = Leap::Tool(pointable);    } else {        Leap::Finger finger = Leap::Finger(pointable);    }

 

Calculate the basic finger position

If you need to calculate the basic position of the finger, you can use the fingertip position and orientation as follows:

    Leap::Vector basePosition = -pointable.direction() * pointable.length();    basePosition += pointable.tipPosition();