In-depth exploration of 3D pickup Technology

-Pan Hong

-March January 2013

-My skills are limited, and mistakes are inevitable. Please give me some advice.

-Email:
Popyy@netease.com

-Weibo.com/panhong101

3D pickup

In the game, players need to select 3D objects by clicking the 2D screen. This process is picking ). Pickup is an essential basic operation for 3D games. It allows players to interact with objects in the game world.

Although the pickup technology is very basic, It puzzles many 3D beginners. Many of my friends have asked me about the pick-up details, which makes it necessary for me to discuss the technology in detail. In fact, the main reason for the complexity of picking up is that it involves multiple stages of the pipeline across domains, and the reverse pipeline is uplinked. In addition, it is a process in which 2D information is extended to 3D. It must be extended and calculated accordingly to obtain correct results. Next we will analyze this technology in detail.

Analysis of Main stages of flow line

Let's take a look at the transformation from the camera space to the viewport:

A vertex v in the camera space enters CVV after perspective transformation. This transformation matrix actually completes two tasks:

1) projects a vertex from a 3D space to a 2D projection plane.

2) the 2D projection points on the projection plane are transformed to the CVV in the same cropping space by linear interpolation.

These transformations are all completed once through the perspective matrix (if you are not familiar with this, please refer to the article "exploring Perspective Projection Transformation ). The reason why I split this step into two steps is that this is very important for analysis.

After the vertex enters the homogeneous cropping space and passes CVV cropping, the perspective division is changed from 4D homogeneous to 3D. After a linear interpolation (encapsulated in the viewport transformation), The point is transformed to the viewport. After the multiple points are raster in the form of triangles, they are seen by the players. The point transformation in the last step can be described as follows:

3) transforms the points in CVV to the viewport through linear interpolation.

After analyzing this transformation process, we know the operations for actually processing the point location information from the camera space, which is the above three steps. In this way, we can first transform the vertex from the viewport back to the projection plane, that is, we can first complete the inverse processing of (2) and (3. Here we do not need to consider the cropping and perspective Division operations, because during the reverse push, the point in the viewport is already an effective point after the cropping, it must be in the CVV, it must be in the projection
Within plane! In addition, switching from viewport to projection plane is always in 2D format.

At the beginning of picking, a player clicks a position on the screen-this is actually clicked in viewport. In response to the player's Click Event, the position of the click in viewport is displayed as P0 (xp0, yp0 ). Then, we linearly interpolation P0 from viewport to CVV to get p1 (xp1, yp1 ):

The above linear interpolation formula (if you are not familiar with the linear interpolation formula, please refer to the linear interpolation section in "Deep Exploration of Perspective Projection Transformation") calculates p1 in CVV in the X direction, the formula in the Y direction is the same. Next, we will change P1 from CVV to projection.
In plane, get P2 (xp2, yp2 ):

The calculation in the Y direction is the same. P2 is the position of the player's clicked point on projection plane in viewport.

Currently, this is good. We have obtained the position of the player clicking on the projection plane in the camera space. But the current point is a 2D point-it is on the projection plane. Players need to pick up a 3D object, so we need to expand 2D information to 3D.

Expand to the 3D world

Extend the 2D point information to the 3D space for picking and use Ray. Ray is a linear model with a fixed end and an infinite extension of the other end. As shown in:

In the camera space, the red line indicates the ray used for picking. Its Fixed End is the eye position (that is, the origin of the camera space), and it passes through the point P2 above the projection plane we just obtained. Rays extend infinitely to space. The first polygon passed through should be the result of picking. In the figure, there are two polygon picking to: green and yellow. The yellow polygon is the first to be passed through, so the result returned by the picking operation is this polygon. In implementation, we generally have two methods to represent a Ray:

struct Ray3D{Point3D m_startingPos;Point3D m_penetratedPos;};struct Ray3D{Point3D m_startingPos;Point3D m_direction;};

The first method indicates the starting point m_startingpos of Ray and any passing point m_penetrated. The second method indicates the starting point m_startingpos and direction m_direction of Ray. These two methods can be easily converted to each other.

After Ray notation, all we need to do is to determine whether Ray has an intersection with each polygon. This is actually a ray-triangle intersection judgment algorithm. This algorithm is common and easy to find. We will not discuss it here. The basic pick-up algorithm (in camera space) is as follows;

extern float ray_triangle_intersection( const Ray3D& ray, const Polygon& polygon );GameObject* picking( const Point3D& P2, const std::vector< GameObject* >& objects ){Ray3D ray;ray.m_startingPos = Point3D( 0, 0, 0 );ray.m_penetratedPos = P2;float minDistance = 10000.0; // Big enoughGameObject* intersected = NULL;for( int i = 0; i < objects.size(); ++i ){GameObject* obj = objects[i];for( int j = 0; j < obj->number_polygons(); ++j ){Polygon* triangle = obj->get_polygon( j );float distance = ray_triangle_intersection( ray, *triangle );if( distance > 0.0 ) // Penetrated{if( distance < minDistance ){minDistance = distance;intersected = obj;}}}}return intersected;}

This brute-force algorithm first generates Ray in the camera space, then traverses all game objects, and traverses every polygon of each game object. It uses the ray_triangle_intersection function to perform Ray cross judgment. If a positive value is returned, it indicates the distance from the ray start point to the interspersed position, and the negative value indicates no interspersed position. The function judges each interspersed polygon and retains the nearest return. If no interspersed polygon exists, null is returned. It is worth mentioning that the optimization of judgment is a problem. Although the ray_triangle_intersection algorithm can be optimized, for a large scenario or model with a large number of polygon instances, traversing all polygon through this method is unacceptable in terms of efficiency. The following two methods are required for optimization:

1) Use the scenario management method and the hierarchical structure method to remove a large number of polygons out of sight in advance. Only the polygon in the line of sight is left.

2) Ray Intersection judgment is performed in units of the surrounding body, rather than a triangle. For example, the box and ball are replaced by Ray and the rectangle and sphere for intersection judgment. A game object can generally be divided into multiple surrounding bodies.

In addition to the above method, there are many other advanced methods that can be used for this optimization. This is usually related to the management method of your game scenario and the 3D object representation method. In particular, if orthogonal projection is used, Ray is not required and can be determined directly on the plane. This degrades to a 2D picking problem.

Back to the world space?

There will be some strategies for the following issues. We need to determine whether picking is performed in the Current Camera space or in the world space. We have processed picking in the camera space, but there is a problem: in the program, we generally do not reserve the position of each 3D object in the camera space, so in this case, we will adopt one of the following two methods:

1) transform Ray into the world space using the inverse camera matrix.

2) convert an object to the camera space using the world matrix and camera matrix for picking, just like the above processing method.

However, even if we adopt 1st methods, we must use the world matrix to transform the model from the model space to the world space. This leads to the picking mode, we need to transform both the model and Ray (unless Ray is always in the model space ). However, in general games, we reserve a "Model-View" matrix, that is, the combination of the world matrix and the camera matrix. In this case, the cost of using the 2nd method is lower than that of the 1st method-ray does not need any transformation. It is worth noting that if the first method is used and the 2nd expression structures of Ray described above are used, m_direction transformation requires reverse placement of the inverse matrix of the camera, this is the same principle as converting the normal of polygon. If you are confused about this problem, you can search for topics related to normal transformation.

Summary

So far, we have discussed the main theoretical methods of 3D picking. Many graphic APIs (such as OpenGL) provide related methods to simplify the picking operation. The picking stage may be different, or some optimizations are made, or the CVV definition is different, while the principle is similar. However, a unified requirement is to have a detailed understanding of the pipeline so that a reasonable solution can be found in the ever-changing needs.