Multi-target tracking by Lagrangian relaxation to min-cost network flow

Multi-target tracking by Lagrangian relaxation to Min-cost network flowhigh-order constraints min-cost network Flow multi- Target tracking

Read "Multi-target tracking by Lagrangian relaxation to min-cost network flow, cvpr,2013" summary.

Last night the boss let a look at this article to write a few words summed up to him, so he looked at, today by the way a brief summary of this article.

The core of this article model is still the network flow algorithm, but unlike the general network flow algorithm is: The general method in the construction diagram directly each observation as node, and the relationship between observation using edge representation, This way each edge represents the degree of similarity or association between node, and in this article the model is built using candidate pair as node, and then pair of candidate pairs exists between this edge, in this manner, It is possible to integrate high-order information between successive three frames, such as a continuous three frames, into the model with approximately constant speed. Because of the different composition, it is necessary to introduce some additional constraints to meet the one-to-one relationship between track-detection in multi-objective tracking. For the proposed model, the appropriate Lagrangian relaxation can be transformed into a general network flow algorithm for efficient solution.

Illustrative overview of proposed graph representation

Use a simple example to clearly illustrate how the model is composed.
Suppose there is now a continuous three-frame image of the observations. The first frame has 3, the second frame has two representations of 4, 5, and the third frame 3 are represented as 6,7,8. The general network flow algorithm is composed as follows (no source and sink points are added here)

Fig1.jpg
Traffic per edge is a binary variable, the network flow should obviously meet the traffic conservation constraints, the cost of each edge is connected to two different frames between the matching degree of observations, then you can use the least cost flow algorithm to solve the model.
The price on each edge of the above model only describes the degree of matching between two frames of connected observations, and the higher-order information in the MTT problem is often more useful. So the author proposed the following composition method

Fig2.jpg
Represents the connection relationship between observations I and J, such asRepresents a connection relationship between observations 1 and 4. A possible match between two consecutive frames as nodes, such as 1, 22, and 2, 32 frames may be matched to nodes, and then the match between different frames if there is a common point, there is an edge between two matches, such as 1, 22 frames between the matchand a match between 2 and 3 framesThere is an edge between. So the cost of each edge is the similarity between the two matches, and the matching information can include the relative speed and apparent difference of the connected observations, so the cost of the edge can contain high-order information of the observations between the connected three frames.
The general hypothesis in MTT (of course, there are many ways to get rid of this constraint): A trajectory can only match one observation in any frame, and the same observation can only correspond to one track. Therefore, the proposed model is to add additional constraints to the nodes to solve the coupling between the nodes, that is, in the color connection nodes, can only select one more, such as, since two have been observation 1, in order to meet the corresponding constraints, must only two selected one.

Problem formulation

Formal presentation model.

The existing length isSequence of images, sectionThe frame hasA observations, whose collection is represented as,Represents the I target of frame K.
A possible matching pair between adjacent frames is a two-tuple, expressed as, these possible matches can be obtained by apparent similarity, distance similarity and so on. The number of possible matches between frame K and k+1 is expressed as, whose collection is represented as。 Then the number of nodes in the whole sequence is, whose total collection is expressed as.

The diagram further refined by Figure 2 is as follows: G= (V,e), where V contains the source point S and the meeting point T, and the two observations of each match link, called incoming node and outgoing node. .
There are two benefits of representing each match as two nodes:
1. Since the maximum traffic per edge is 1 and the flow balance is constrained, the traffic leaving the outgoing point can only be 1 because there is only one entry link
2. This allows the unary and binary constraints in the general network stream algorithm to be added directly to the link inside the match, and the higher-order information is placed between match and match deges.

Fig3.jpg
Note that this is a continuous 3-frame image, starting with the emphasis of 3 frames just to fuse higher-order information. Like occlusion, this problem is not necessarily a continuous frame, which can be solved by constructing a similar graph from a discontinuous frame.

The entire model is represented as follows

This represents the cost of the Edge ij, (1) represents the minimum cost, (2) represents a binary constraint, (3) represents a traffic balance constraint, and (4) represents an additional constraint for one-to-one correspondence. (1) (2) (3) is a general network flow algorithm model, for the constraint (4), The first S is composed of outgoing and incoming points coincident matches composition, the whole sequence of a total of Q this set.

In order to solve the model, the constraint (4) is put into the target by Lagrange relaxation, and then it can be converted into a general network flow algorithm model for solving.

Which represents the Lagrange multiplier

Stopping criteria

Because some constraints can be too strong and always impossible to satisfy, the iterative process may never converge, which is the way to terminate the algorithm by limiting the maximum number of iterations.
The results of the iteration are further processed:

1. Connect the selected matches to make tracks

2. Bring the conflicting track out and put it in a list of "competing tracks"

3. Select the track of lowest cost in the conflicted tracks as the correct track out

4. For the remainder of the conflicted tracks tracks, the match to eliminate the conflict to see whether it can still meet the conditions of the trajectory, such as smooth, length, etc., meet the creation of a new trajectory, not satisfied to throw away.

Experiments

Experiments were carried out on the Psu,tud and ETHMS databases, and the results of the experiments were described in this paper.

Conclusion

1. The model uses higher-order information relative to the general network flow algorithm
2. But the higher order here is only a 3-order message, and now there are some ways to use higher-order information than a multi-target tracking based on the approximation of tensor rank.
3. The model can be effectively transformed into a general network flow algorithm by Lagrange relaxation.
4. In the case of non-convergence of the algorithm, a greedy algorithm is used as a complement to the end of the forced algorithm.

Multi-target tracking by Lagrangian relaxation to min-cost network flow