Dynamic Programming algorithm

First, the basic concept

The dynamic planning process is that each decision relies on the current state and then causes the state to shift. A decision sequence is generated in the state of change, so the process of solving the problem by this multistage optimization decision is called dynamic programming.

Second, basic ideas and strategies

The basic idea is similar to the divide-and-conquer method, and the problem to be solved is decomposed into several sub-problems (stages), and the solution of the sub-stage is solved in order, which provides useful information for the solution of the latter sub-problem. When solving any sub-problem, the various possible local solutions are listed, and the local solutions that are likely to achieve the best are preserved by decision, and other local solutions are discarded. Solve each sub-problem in turn, the last sub-problem is the solution of the initial problem.

Because the problem of dynamic planning solves most overlapping sub-problems, in order to reduce the repetition, we only solve each sub-problem once and save the different states of different stages in a two-dimensional array.

The biggest difference between the division and the method is: suitable for the problem solved by the dynamic programming method, the sub-problems obtained after decomposition are often not independent of each other (that is, the next sub-stage solution is based on the solution of the previous sub-stage, and further solution).

Third, the application of the situation

There are 3 properties that can be used to solve the problem of dynamic programming:

(1) Optimization principle: If the optimal solution of the problem contains sub-problem solution is also optimal, it is said that the problem has the optimal sub-structure, that is, to meet the optimization principle.

(2) No effect: that is, once a stage state is determined, it is not affected by the decision after this state. In other words, the subsequent process of a State does not affect the previous state, only the current state.

(3) There are overlapping sub-problems: That is, sub-problems are not independent, a sub-problem in the next stage of decision-making may be used more than once. ( This nature is not a necessary condition for dynamic programming, but without this nature, the dynamic programming algorithm has no advantage over other algorithms)

Iv. Basic steps of the solution

The problem that dynamic programming deals with is a multi-stage decision-making problem, which usually starts from the initial state and reaches the end state through the choice of the intermediate stage decision. These decisions form a sequence of decisions, while defining an active route to complete the process (usually the optimal active route). The design of dynamic planning has a certain pattern, which usually goes through the following steps.

Initial state →│ decision 1│→│ decision 2│→ ... →│ decision n│→ End State

Figure 1 Dynamic planning decision process

(1) The dividing stage : According to the question Time or the space characteristic, divides the question into several stages. In the partitioning phase, note that after the division of the stage must be ordered or sortable, otherwise the problem can not be solved.

(2) determine state and state variables : The various objective situations in which the problem is developed into various stages are expressed in different states. Of course, the choice of state to meet no-no validity.

(3) determine the decision and write out the state transition equation : Because decision-making and state transfer have a natural connection, State transfer is to export the state of this stage according to the state and decision of the previous stage. So if the decision is made, the state transfer equation can be written out. In fact, it is often done in turn to determine the decision-making method and the state transition equation based on the relationship between the states of the adjacent two phases.

(4) looking for boundary conditions : The given state transition equation is a recursive type that requires a recursive termination condition or boundary condition.

In general, the state transition equation (including boundary conditions) can be written as long as the phase, State and state transfer decisions of the problem are resolved.

Practical applications can be designed in a few simplified steps, as follows:

(1) Analyzing the properties of the optimal solution and characterizing its structural characteristics.

(2) A recursive definition of the optimal solution.

(3) Calculate the optimal value from the bottom-up or top-down memory (Memo method)

(4) According to the information obtained when calculating the optimal value, the optimal solution of the structural problem

Five, the implementation of the description of the algorithm

The main difficulty of dynamic programming is the theoretical design, that is, the above 4 steps to determine, once the design is complete, the implementation of the part will be very simple.

Using dynamic programming to solve problems, the most important thing is to determine the three elements of dynamic planning:

(1) phase of the problem (2) status of each stage

(3) the recurrence relationship between the previous phase and the latter one.

     Recursive relationship must be from the minor problem to the transformation between the larger problem, from this point of view, dynamic planning can often be implemented with recursive procedures, but because recursion can make full use of the solution of the previously saved sub-problems to reduce the repetition, so for large-scale problems, there is a recursive incomparable advantage, which is the core of the dynamic programming algorithm.

The three elements of dynamic programming are determined, the whole process can be described by an optimal decision table, the Optimal decision table is a two-dimensional table, where the row represents the decision stage, the column indicates the problem state, the table needs Fill in the data generally corresponds to the problem at some stage of the optimal value of a state (such as the shortest path, the longest common sub-series, the maximum value, etc.), the process of filling out is based on a recursive relationship, starting from 1 rows and 1 columns, in the order of row or column priority, fill in the form, Finally, the optimal solution of the problem is obtained by simple trade-offs or calculations based on the data of the whole table.

F (n,m) =max{f (N-1,m), F (N-1,m-w[n]) +p (n,m)}

VI. basic framework of dynamic programming algorithms

1 Code2  for(j=1; j<=m; j=j+1)//First Stage3XN[J] =initial value;4 5   for(i=n-1; i>=1; i=i-1)//other stages of n-16     for(j=1; J>=f (i); j=j+1)//f (i) expressions in relation to I7Xi[j]=j=max (or min) {g (xi-1[J1:j2]), ..., G (xi-1[jk:jk+1])};8 9t = g (x1[j1:j2]);//A scheme for solving the optimal solution of the whole problem by the optimal solution of sub-problemTen  One print (x1[j1]); A  -  for(i=2; i<=n-1; i=i+1) - {   thet = t-xi-1[Ji]; -  -       for(j=1; J>=f (i); j=j+1) -         if(t=Xi[ji]) +               Break; -}

View Code

Dynamic Programming algorithm