Dynamic Programming algorithm

R.bellman and others in 1951 in the study of multi-stage decision-making process optimization problems created a new method to solve such process optimization problems.

The inverse recursive method is called the Dynamic Programming Method (programming).

Multi-stage decision making

There is one type of problem that can break down its activity process into several interconnected phases that require decision-making at every stage of the process to achieve the best possible activity. The decision choices made at each stage depend on the current state and affect the future development. When the decision of each stage is determined, a decision sequence is formed, which determines an active route of the whole process. This kind of problem is regarded as a multi-stage decision which is interrelated and has a chain structure and is called multistage process.

Typically, in a multi-stage decision process, there are multiple decision sequences at a certain stage, if the decision is made with the following principles: The solution is bottom-up (that is, from the endpoint to the starting point), and the selection of each step always depends on the previous selection, and this step only excludes the impossible decision sequences.

Dynamic programming attention is applied to solving optimization problems. This type of problem may have multiple possible solutions, each with a value, and dynamic programming is the solution to find the optimal (maximum or minimum) value of the solution. If there are several solutions to the optimal value, only one of them is taken.

　　

The dynamic programming is similar to the divide-and-conquer method and greedy method, which decomposes the original problem into several smaller, similar sub-problems, and produces a global optimal solution by solving sub-problems. The differences are as follows:

(1) When using the greedy method, the current selection may depend on all the choices that have been made, but not on the choices and sub-issues to be made. So the greedy method is from top to bottom (that is, from the starting point to the end point), make the greedy choice step by step. Of course, if the current selection may depend on the solution of sub-problem, it is difficult to achieve global optimal solution by local greedy strategy.

(2) When using the divide-and-conquer method, the sub-problems decomposed by the original problem are usually independent of each other, that is, they do not contain the common sub-problems, so once the solution of each sub-problem is obtained recursively, the solution of each sub-problem can be merged into the solution of the problem from bottom to back. If the sub-problems are not independent of each other, then the division of the law to make a lot of unnecessary work, the repetition of the common sub-problem.

(3) Dynamic programming allows for the interdependence of sub-problems broken down by the original problem. Each sub-problem is solved only once and the result is saved to avoid repeating the calculation every time the problem is encountered.

To sum up, the basic idea of dynamic programming is to decompose the problem to be solved into several interrelated sub-problems, solve the sub-problems first, and then get the solution of the original problem from the solution of these sub-problems. For a recurring sub-problem, it is solved only at the first encounter, and the intermediate calculation results are saved, and the intermediate results are directly referenced in the future, without having to be re-solved.

Conditions for dynamic planning

(1) Optimization principle (optimal sub-structural properties)

If the optimal solution of the problem contains the solution of the sub-problem is also optimal, it is said that the problem has the best sub-structural properties, that is, to meet the optimization principle.

The so-called optimization principle is that regardless of the previous strategy, the decision thereafter must be based on the current state (resulting from the previous decision) of the optimal decision. Because some recursion for some problems does not necessarily guarantee the optimal principle, it is necessary to validate the problem when it is solved. If the optimal principle cannot be maintained, the dynamic programming method can not be used to solve it. After getting the recursive formula of the optimal solution, we need to perform backtracking to construct the optimal solution.

(2) No effect

An important condition for applying dynamic programming is to arrange the stages in a certain order, and the state of phase I can only be determined by the state of the stage i+1, which is not related to the other States, especially to the non-occurrence state. Each state is a "complete summary of past history".

(3) overlap of sub-problems

The overlap of sub-problems is that when solving the problem from top to bottom by recursive algorithm, the sub-problem is not always a new problem, and some sub-problems may be counted repeatedly. The dynamic programming method is to use this overlapping property of sub-problem, only once for each sub-problem, then save the result of calculation, when we need to calculate the already calculated sub-problem, it is simple to look at the previous calculation results, so as to obtain high efficiency of solving problems.

The overlap of sub-problems is not a necessary condition for dynamic programming, and if this condition is not satisfied, dynamic programming has no advantage over other algorithms.

The steps of the dynamic programming method to solve the problem

(1) Analysis: The original problem is analyzed to find out the structural characteristics of the optimal solution of the problem.

(2) Decomposition: The time and space characteristics of the given problem are decomposed into several interrelated phases, and the recursive relation of calculating local optimal solution is determined, which is the key and difficult point to solve the problem by using the dynamic programming method. It is important to note that the various stages after decomposition must be orderly or sortable, i.e., no-back sex. Otherwise, the problem cannot be solved with dynamic rules.

(3) Solve: For each stage through the bottom-up method to obtain the local problem of the optimal solution. A recursive termination condition or boundary condition is required.

(4) Merging: The solutions obtained from each stage are combined into the solution of the original problem, that is, to construct an optimal solution.

The whole solution can be described by a two-dimensional array of the optimal decision table, where the row represents the stage of the decision, the column represents the state of the problem, and the data that the table needs to fill in generally corresponds to the optimal value of the problem in some stage state, such as the shortest path, the longest common subsequence, and the maximum value. The fill-in process is based on a recursive relationship starting from 1 rows and 1 columns, and filling in the table in order of row or column precedence. Finally, the optimal solution of the problem is obtained by simple trade-offs or calculations based on the data of the whole table.

The key to dynamic programming is to solve redundancy, which is a space-time-changing technology, so it has more space complexity than other algorithms.

Examples of application of dynamic programming methods:

Multi-Source Shortest path

For a given non-bearing network graph, the shortest path between any two nodes is calculated. There are usually two methods, one is to call the N-times Dijkstra algorithm with each vertex in the graph as the source point, and the other is to use the floyed algorithm. The time complexity of both methods is O (N*n*n), the latter is relatively simple.

http://blog.csdn.net/wangdd_199326/article/details/62235750

Dynamic Programming algorithm