Dynamic Planning (BASICS)

1.What is dynamic planning:

Dynamic Planning, like the division and Control Law, solves the entire problem by combining sub-problems. But the difference is that the sub-GovernanceAlgorithmIt refers to dividing a problem into several independent subproblems, solving each subproblem recursively, and then merging the subproblems to obtain the original problem solution. Dynamic Planning is applicable to subproblems that are not independent, that is, all subproblems include public subproblems. Dynamic Planning solves each subproblem only once and saves the result in a table, so as to avoid re-calculating the answer every time a subproblem occurs.

 

2.Several important features of dynamic planning:

2.1Optimal sub-structure:

The first step in finding the optimal problem with dynamic planning is to describe the structure of the optimal solution. If an optimal solution of a problem contains the optimal solution of a subproblem, the problem has an optimal sub-structure. When a problem has an optimal sub-structure, it prompts that dynamic planning may be applicable (note that in this case, greedy policies may also apply). In dynamic planning, we use the optimal solution of the sub-problem to construct an optimal solution of the problem. For example, if f [I] [J] indicates that the first I items are considered, the maximum size of the items in the backpack is J, then f [I] [J] = max {f [I-1] [J], F [I-1] [J-V [I]}, if f [I-1] [J] or F [I-1] [J-V [I] is not optimal, then f [I] [J] is not optimal, this is the optimal sub-structure.

 2.2Replay questions:

The so-called overlap subproblem, that is, when a recursive algorithm constantly calls the same problem, we say that the optimal problem includes the overlap subproblem. In this case, we propose two solutions, 1. Memory-based search (that is, the memorandum algorithm); 2. Bottom-up recurrence. This is also the key to DP thinking. For details, see the example of digital triangle (see Liu rujia's algorithm competition getting started classic), repeated calculation of D (), D (), and D.

 2.3Status:

Status is a very important concept. If the algorithm passes through, more than half of the dynamic planning has been completed when the status of a successful question is designed. In fact, if a real problem can be identified and solved using the DP method, and the status can be defined, the problem will suddenly become open. The next step is to find the coarse state transition equation.

 2.4State transition equation:

The state transition equation is the core of the dynamic planning algorithm and the key to implementing the algorithm. The status of the current stage in dynamic planning is often the result of the status of the previous stage and decision-making of the previous stage. That is, the formula used from one state to another. For example, F [I] [J] = max {f [I-1] [J], F [I-1] [J-V [I]} In a 0-1 backpack.

 

3.When to use dynamic planning:

Dynamic Planning is usually applied to optimization problems. This type of problem may have many feasible solutions, each of which has a value, and we want to find a solution with an optimal (maximum or minimum) value. This problem is called the "one" Optimal Solution (rather than the "definite" optimal solution) of the problem, because there may be multiple values that take the optimal solution.

The basic conditions for dynamic planning are:

1) optimal substructure

2) subproblem overlapping nature

When applying dynamic planning, for repeated sub-problems, you only need to solve the problem for the first time and save the answer so that you can reference it directly later without having to solve it again, this greatly improves the efficiency of solving problems. In contrast, the general search technology, whether or not a subproblem has been solved, will solve it again as long as it is met, thus affecting the efficiency of understanding the question.

 

4.Two methods for implementing the dynamic planning algorithm:

Description of the digital triangle problem:There is a triangle composed of non-negative integers. The first row has only one number, except for the bottom row, each of which has a number in the lower left and lower right.

1

3 2

4 10 1

4 3 2 20

4.1Memory-based search:

Int D (int I, Int J)

{

If(D [I] [J]> = 0) Return d [I] [J];

Return d [I] [J] = A [I] [J] + (I = n? 0: d [I + 1] [J]>? D [I + 1] [J + 1]);

}

The so-called memory-based search is recorded if it is calculated as d [I] [J]. It can be used directly the next time d [I] [J] is used. This is also the main difference between using DP and recursive solutions.

4.2Recursive calculation:

Int I, J;

For (j = 1; j <= N; j ++) d [N] [J] = a [n] [J];

For (I = n-1; I> = 1; I --)

{

For (j = 1; j <= I; j ++)
{

D [I] [J] = A [I] [J] + d [I + 1] [J]>? D [I + 1] [J + 1];

}

}

The recursion here is actually similar to the memory-based search, but the recursive algorithm is bottom-up and needs to clarify the computing sequence of each State. However, the memory search should record whether each state has been counted.

 

5.Examples of Dynamic Planning Algorithms:

Digital triangle: Typical Example

0-1 backpack problem: multi-stage decision-making Problem

Dynamic Planning On dag: model the problem to the graph and use dynamic planning

 

6.Notes for using dynamic planning:

6.1 initial value problems:

6.2 State Transfer Process