Summary of five common algorithm design methods for "reprint" algorithm design

Reprinted from http://blog.csdn.net/zolalad/article/details/11393915

A summary of the five common algorithms design methods for algorithm design

I. "Divide and Conquer law"

In computer science, divide-and-conquer method is a very important algorithm. The literal explanation is "divide and conquer", which is to divide a complex problem into two or more identical or similar sub-problems, then divide the problem into smaller sub-problems ... Until the last sub-problem can be solved simply, the solution of the original problem is the merger of the solution of the sub-problem.  This technique is the basis for many efficient algorithms , such as sorting algorithms (fast sorting, merge sorting), Fourier transform (Fast Fourier transform) ... such as The computational time required for any problem that can be solved with a computer is related to its size. The smaller the problem, the easier it is to solve it directly, and the less computational time it takes to solve it. For example, for the ordering of n elements, when n=1, no calculations are required. N=2, the order can be sorted once compared. N=3 only 3 times to compare, .... And when n is large, the problem is not so easy to deal with . It is sometimes quite difficult to solve a problem of a larger scale directly. 

The design idea of divide and conquer method is to divide a big problem which is difficult to be solved directly into some smaller problem in order to  conquer, divide and conquer .

Divide-and-conquer strategy is: for a size of n, if the problem can be easily solved (for example, the size of n smaller) is directly resolved, otherwise it is divided into K small sub-problem, these sub-problems are independent of each other and the original problem form , the recursive solution of these sub-problems, and then  The solution of each sub-problem is combined to get the solution of the original problem . This algorithm design strategy is called divide-and- conquer method . 

If the original problem can be divided into K (1<k≤n) sub-problem, and these sub-problems can be solved and can use the solution of these sub-problems to find out the solution of the original problem, then this kind of partition method is feasible. The sub-problems produced by the Division method are often the smaller models of the original problems,  which makes it convenient to use "recursive technology" . In this case, the repeated application of the method of separation, can make sub-problems and the original problem type consistent and its scale is shrinking, and finally make the sub-problem to easily find its solution, which naturally leads to the generation of recursive process. "  Divide and conquer" and "recursion "  like a pair of twin brothers, often at the same time applied in the algorithm design, and thus produce many efficient algorithms . 

The problems that can be solved by the method of division and administration generally have the following characteristics:

1) The scale of the problem is reduced to a certain extent and can be easily resolved

2) The problem can be decomposed into several small-scale same problems, that is, the problem has the best substructure properties. (prerequisite) 

3) The solution of sub-problems decomposed by this problem can be combined into the solution of the problem; (the most critical point)

4) The problem is separated from each other sub-problems, that is, the sub-problem does not include the common sub-sub -problem.

The first feature of the above is that most of the problems can be satisfied, because the computational complexity of the problem is generally increased with the increase of the size of the problem; The second feature is the premise of applying the divide-and-conquer method. It is also the most problems can be satisfied, this feature reflects the application of recursive thinking; The third feature is the key, whether the use of divide-and-conquer method depends entirely on whether the problem has a third feature, if the first and second features, and does not have a third feature, you can consider using greedy or dynamic programming method. The fourth feature relates to the efficiency of division and treatment , if the sub-problems are not independent, then divide and conquer the law to do a lot of unnecessary work, repeated to solve the common sub-problem, although the use of divided treatment method, but generally with the dynamic programming method is better. 

The basic steps of divide and conquer law

The divide-and-conquer method has three steps on each level of recursion :

1) decomposition: The original problem is divided into a number of small, independent, and the original problem form the same sub-problem;

2) solve: If the sub-problem is small and easy to solve the direct solution, or recursively solve each sub-problem

3) Merging: The solution of each sub-problem is merged into the solution of the original problem. 

Its general algorithm design pattern is as follows:

Divide-and-conquer (P)

If | P|≤n0 {

Then     return (Adhoc (P))

}else{

         The P continues to be decomposed into smaller sub-problems P1, P2,..., Pk

     For I←1 to K

Do  yi←divide-and-conquer (pi) recursive solution pi

     T←merge (Y1,y2,..., yk) merging sub-issues

}

Return (T)

Among them | P| indicates the size of the problem p, N0 is a threshold value, indicating that when the size of the problem p does not exceed n0, the problem is easily solved and no further decomposition is necessary. Adhoc (P) is the basic sub-algorithm in the division method, which is used to solve the problem P of small scale directly. Therefore, when the scale of P does not exceed n0, it is solved directly with the algorithm Adhoc (p). The algorithm merge (Y1,y2,..., YK) is the merging sub-algorithm in the division method, which is used to P1 the sub-problem of P, P2 the corresponding solution y1,y2,..., YK of,..., PK to the solution of P.

Application of divide-and-conquer method: 1. The problem of recursion and Hanoi Tower, 2. Two method to find the approximate solution of Equation 3. Implementing merge sorting in C + + 4. Divide-and-conquer algorithm for maximal and minimum values

Analysis of the complexity of division and treatment method

The recursive equation and its solution give only the value of T (n) when n equals m operational, but if T (n) is considered to be smooth enough, the value of T (N) can be estimated at the speed of T (N) at the operational of N equals M. It is generally assumed that T (N) is monotonically ascending, thus when mi≤n<mi+1, T (MI) ≤t (n) <t (mi+1).

Second, "Dynamic planning Method"

The principle of optimization

 1951 American mathematician R. Bellman and others, according to a class of   multi-stage problem  features, the "  Multi-stage decision  "problem changed to  a series of  interconnected  single-stage issues ",  then  Resolve the  one by one. and   Some static models, as long as the artificial introduction of "time" factor, divided into time, can be transformed into a multi-stage dynamic model, using dynamic programming method to deal with the  . At the same time, he proposed the "  Optimization Principle " (Principle of optimality) to solve such problems: "The optimal decision of a process has such properties: that is, regardless of its initial state and initial decision, Its future strategies must constitute the optimal strategy for the process of the initial state of the state formed by the first decision. In a nutshell, the sub-strategy of an optimal strategy must be optimal for its initial and final state.   This "optimization principle" if  To describe   in a mathematically-based language, that is:  Assuming that in order to solve an optimization problem, we need to make N decision D1,D2,...,DN in turn, if the decision sequence is optimal, For any integer k,1 < K < N, no matter what the preceding K-decision is, the subsequent optimal decision will depend only on the current state determined by the previous decision, i.e. the subsequent decision DK+1,DK+2,...,DN is also optimal. 

optimization principle is the foundation of dynamic programming , any problem, if lost the support of this optimization principle, it is impossible to use dynamic programming method to calculate. The problems that can be solved by dynamic programming need to meet certain conditions:

(1) The state of the problem must satisfy the optimization principle;

(2) The state in the problem must meet no-no validity.

The so-called no-effect refers to the following: "The state of the next moment is only relevant to the current state, and not to the state before the current state, the current state is a summary of previous decisions".

Problem Solving mode

The problem that dynamic programming deals with is a multi-stage decision-making problem, which usually starts from the initial state and reaches the end state through the choice of the intermediate stage decision. These decisions form a sequence of decisions, while defining an active route to complete the process (usually the optimal active route). The design of dynamic planning has a certain pattern, which usually goes through the following steps.

Initial state →│ decision 1│→│ decision 2│→ ... →│ decision n│→ End State

Figure 1 Dynamic planning decision process

(1) Division stage: According to the question Time or the space characteristic, divides the question into several stages. In the partitioning phase, note that after the division of the stage must be ordered or sortable, otherwise the problem can not be solved.

(2) determining state and state variables: The various objective situations in which the problem is developed into various stages are expressed in different states. Of course, the choice of state to meet no-no validity.

(3) Determine the decision and write out the state transition equation: because decision-making and state transfer have a natural connection, state transfer is to export the state of this stage according to the state and decision of the previous stage. So if the decision is made, the state transfer equation can be written out. In fact, it is often done in turn, depending on the relationship between the two adjacent paragraphs to determine the decision.

(4) Finding the boundary condition: the given state transfer equation is a recursive type, which requires a recursive terminating condition or boundary condition.

Application of Dynamic programming Method: 1. Dynamic planning for 0/1 knapsack problem   3. Using dynamic programming to achieve missile interception 4. Maximizing Return on Investment

Algorithm implementation

The main difficulty of dynamic programming is the theoretical design, that is, the above 4 steps to determine, once the design is complete, the implementation of the part will be very simple. using dynamic programming to solve problems, the most important thing is to determine the three elements of dynamic planning: the phase of the problem, the state of each phase and the recurrence relationship between the previous phase and the latter. The recursive relationship must be from the minor problem to the transformation between the larger problem, from this point of view, dynamic planning can often be implemented with recursive programs, but because recursion can take full advantage of the previous saved sub-problem of the solution to reduce duplication, so for large-scale problems, there is a recursive incomparable advantage, This is also at the heart of the dynamic programming algorithm. The three elements of dynamic programming are determined, the whole process can be described by an optimal decision table , the optimal decision table is a two-dimensional table, where the row represents the stage of the decision, the column represents the state of the problem, The table needs to fill in the data generally corresponds to this problem in a certain state of the optimal value (such as the shortest path, the longest common sub-series, the maximum value, etc.), the process of filling out is based on the recurrence of the relationship, starting from 1 rows and 1 columns, in the order of row or column priority, fill in the form, Finally, the optimal solution of the problem is obtained by simple trade-offs or calculations based on the data of the whole table. The following is an example of solving the problem by using dynamic programming algorithm to solve the problem of maximizing investment return and the longest common sub-sequence.

Three, "Greedy algorithm"

The so-called greedy algorithm refers to, in the problem solving, always make the best choice at present. In other words, not considering the overall optimality, he makes only the local optimal solution in a certain sense. The heart algorithm is not able to get the whole optimal solution for all the problems, but it can produce the whole optimal solution or the approximate solution of the whole optimal solution for many problems with wide range. 

The basic idea of greedy algorithm is as follows:

1. Build a mathematical model to describe the problem.

2. Divide the problem of solving into several sub-problems.

3. To solve each sub-problem, the local optimal solution of sub-problem is obtained.

4. The solution of the problem is solved by the local optimal solution to the original solution problem.

The process of implementing the algorithm:

Starting from an initial solution of a problem;

While you can move forward one step toward a given total target

Finding a solution element of a feasible solution;

A feasible solution to the problem of the combination of all solution elements;

Here is a greedy algorithm can use the solution of the problem, greedy solution is really good, but not the optimal solution:

Example Analysis:

[Knapsack problem] There is a backpack, the backpack capacity is m=150. There are 7 items that can be divided into any size.

It is required to maximize the total value of the items loaded into the backpack, but not to exceed the total capacity.

Item: A B C D E F G

Weight: 35 30 60 50 40 10 25

Value: 10 40 30 50 35 40 30

Analysis:

Objective function: ∑pi max

The restriction is that the total weight of the loaded item does not exceed the backpack capacity: ∑wi<=m (m=150)

(1) According to the greedy strategy, each time to select the most valuable items loaded into the backpack, the results are optimal?

(2) Can the optimal solution be obtained for each item with the smallest weight selected?

(3) Each time the unit weight value of the most valuable items, become the solution of the strategy.

It is worth noting that the greedy algorithm is not completely not available, once the greedy strategy has been proved , it is an efficient algorithm. greedy algorithm is also a very common algorithm, this is because it is simple, the construction of greedy strategy is not very difficult. Unfortunately, it needs to be proven before it can really be used in the algorithm of the problem. 

In general, the proof of greedy algorithm revolves around: the optimal solution of the whole problem must be obtained by the optimal solution of the sub-problem in the greedy strategy.

For the 3 kinds of greedy strategies in the example, it is impossible to set up ( can not be proved), explained as follows:

(1) Greedy strategy: Choose the most valuable person. Counter Example:

W=30

Item: A B C

Weight: 28 12 12

Value: 30 20 20

According to the strategy, first select Item A, then you can no longer select, however, choose B, C is better.

(2) Greedy strategy: Choose the smallest weight. Its inverse example is similar to the first strategy's counter example.

(3) Greedy strategy: Select items with the greatest value per unit weight. Counter Example:

W=30

Item: A B C

Weight: 28 20 10

Value: 28 20 10

According to the strategy, three items per unit weight value, the program can not be judged according to the existing policy, if you choose a, the answer is wrong.

Greedy algorithm application: 1.     kruskal algorithm       of minimum spanning tree for Prim algorithm of minimum spanning tree

The application of greedy algorithm in Knapsack 4.  Greedy algorithm of automobile refueling problem 

Iv. "Backtracking Method"

The backtracking method is an optimal search method, which is searched forward according to the selection criteria to achieve the goal. However, when the exploration of a step, found that the original choice is not good or not reach the goal, then return to the one-step re-selection, such a failure to return to go back to the technology as backtracking, and meet the backtracking condition of a state point of the "backtracking point."

1. General description of Backtracking method

The problem p, which can be solved by backtracking, is usually expressed as: a state space of known n-tuple (X1,X2,...,XN) e={(X1,X2,...,XN) ∣xi∈si, i=1,2,...,n}, given a constraint set D for a component in an n-tuple All n tuples that satisfy all the constraints of D in E are required. Where SI is the definition domain of the Component XI, and | si| Limited, I=1,2,...,n. We call any n-tuple in E that satisfies all the constraints of D as a solution to the problem p.

The simplest method of solving the problem p is the enumeration method, that is, all n-tuples in e are individually detected to satisfy all the constraints of D and, if satisfied, a solution to the problem p. But obviously, the amount of computation is quite large.

We find that for many problems, the given constraint set D is complete, that is, I-tuple (X1,X2,...,XI) satisfies all constraints in D that involve only x1,x2,...,xi, meaning that J (j<i) tuples (X1,X2,...,XJ) must also satisfy only x1,x2 in D, ..., All of XJ's constraints, I=1,2,...,n. In other words, any n-tuples prefixed with (X1,X2,...,XJ) (x1,x2,...,xj,xj+1, ..., X1,X2,...,XJ), as long as there is a 0≤j≤n-1, to violate one of the constraints in D that is only related to X1,X2,...,XJ; xn) must also violate a constraint in D that relates only to X1,x2,...,xi, N≥i>j. Therefore, for the problem of completeness of constraint set D p, once the detection determines that a J-tuple (X1,X2,...,XJ) violates a constraint in D that only involves X1,X2,...,XJ, it can be affirmed that any n-tuple (x1,x2,...,xj,xj+1, prefixed with (X1,X2,...,XJ)) ..., xn) are not the solution to the problem p, so you do not have to search for them and detect them. Backtracking is a more efficient algorithm than enumeration method, which is based on the above-mentioned nature of this kind of problem.

The Backtracking method first represents the state space E of the N-tuple of the problem p as a weighted ordered tree with a height of n, and converts all the solutions of the problem p in E into all solutions of the problem p in T. Tree T is similar to a search tree, it can be constructed as follows: The elements in SI can be arranged in XI (1), Xi (2), ..., Xi (mi-1), | si| =mi,i=1,2,...,n. At the beginning of the root, I have mi son for each of the nodes in the first layer of T. This mi son to the side of their parents, according to the order from left to right, respectively, with the right xi+1 (1), xi+1 (2), ..., Xi+1 (mi), i=0,1,2,...,n-1. In this way, an n-tuple (X1,X2,...,XN) in e corresponds to a leaf node in T, and the right of the root of T to the N-side on the path of the leaf node is x1,x2,...,xn respectively, and vice versa. In addition, for an arbitrary 0≤i≤n-1,e in the N-tuple (X1,X2,...,XN) of a prefix i-tuple (X1,X2,...,XI) corresponds to a non-leaf node in T, the root of T to the path of the non-leaf node in order I edge of the right is x1,x2,...,xi, and vice versa. In particular, an unprecedented prefix () of any n-tuple in E, corresponding to the root of T.

Thus, a solution for finding the problem P in E is equivalent to searching for a leaf node in T, which requires n weights corresponding to the N-bars of the x1,x2,...,xn from the root of T to the path of the leaf node to satisfy all constraints of the constraint set D. In T search for the required leaf node, it is natural to start from the root, the depth-first strategy to step into, that is, to search for the constraints of the prefix 1 tuples (x1i), prefix 2 tuples (x1,x2) 、..., prefix i-tuple (x1,x2,...,xi), ..., until i=n.

In backtracking, the tree referred to above is called the state space Tree of problem p; any node on the tree T is called the state node of the problem p; any leaf node on the tree T is called a solution state node of the problem P Any one of the leaf nodes on the tree t that satisfies all constraints of the constraint set D is called a response state node of question p, which corresponds to a solution of the problem p.

General steps to solve problems with backtracking:

(1) To define the solution space for the problem;

(2) Determine the spatial structure of the solution that is easy to search;

(3) Searching the solution space in depth first, and using pruning function to avoid invalid search during the searching process.

3. Backtracking method to solve the minimum number of car refueling problems 4. Finding out the total number of R in N natural numbers by backtracking method

V. "Branch and Gauge" method

Basic idea:

In the branch-and-gauge approach, each Slipknot point has only one chance to become an extension node. Once the Slipknot point becomes an extension node, all its son nodes are generated at once. In these sons ' nodes, the node of the son that causes the non-optimal solution or causes the Slipknot is discarded, and the remaining son nodes are added to the table of points. Thereafter, removing a node from the Slipknot point table becomes the current expansion node and repeats the above node expansion process. This process persists until you find the desired solution or the Slipknot point table is empty.

Two common branch-and-gauge methods:

(1) Queue-type (FIFO) branch-and-gauge method

(2) Priority queue-Type branch-gauge method

The node with the highest priority is selected as the current expansion node according to the priority specified in the priority queue.

The difference between the branch-bound method and the backtracking method:

(2) different ways of searching: backtracking searches the solution space tree in a depth-first way, while the branch-bound rule searches the solution space tree in breadth-first or least-cost-first way.

Dynamic search of the solution space tree

(1) Backtracking solution 0/1 knapsack problem, although pruning reduces the search space, but the entire search by the depth of the first mechanical, is blind search (can not predict the nodes below the node how to do).

(2) The backtracking solution of the TSP is also blind (although there are objective functions, but only after finding a workable solution to make sense)

(3) Branch-bound method first determines a reasonable limit function, and according to the bounds of the function to determine the target function bounded [down, up]; then follow the breadth-first strategy to traverse the problem of the solution space Tree, on a branch, search the node in turn all the child nodes, Estimate the possible values of the target function of these children's nodes (to minimize the problem, to estimate the node's down, to maximize the problem, and to estimate the node's up). If the target function value of a child's node is outside the bounds of the target function, it is discarded (the solution generated from this node will not be better than it is currently), otherwise it will be in the pending table.

The design idea of branch and gauge method

To solve maximization problem, the solution vector is x= (x1,..., xn), and the value range of Xi is si,| Si|=ri. When searching the solution space tree of the problem using the branch-bound function, the bounds of the target function are estimated according to the bounded functions (down, up), and then the R1 children nodes of the root node are extended from the root node, thus constituting the possible value of R1 of the component X1.

The R1 child node estimates the possible objective function bound (x1), which means that all possible values of a subtree with this node root are not more than bound (x1), i.e.:

Bound (x1) ≥bound (x1,x2) ≥ ... ≥bound (x1,..., xn)

If the target function value of a child's node is outside the lower bound of the target function, the child node is discarded, otherwise the child node is saved in the Pending node table Pt.

Then take the maximal node of the target function in the PT table as the root node of the extension, repeat the above.

The feasible solution to a leaf node is x= (x1,..., xn), and the target function value bound (x1,..., xn).

Application of branch-and-gauge method:1.

3. Branch Clearance Method 0 1 knapsack problem 4. The problem of traveling salesman with branch and gauge method

Collation reference:http://c.chinaitlab.com/special/algorithm/Index.html

Summary of five common algorithm design methods for "reprint" algorithm design