Dynamic Planning Algorithm Analysis

I. optimization of multi-stage decision-making process

-- Basic model of dynamic planning

In real life, there is a kind of activity process. Due to its particularity, the process can be divided into several interrelated stages, and decision-making is required at each stage of the process, so that the entire process can achieve the best activity effect.
Result. Therefore, the decision-making in each stage cannot be determined arbitrarily. It depends on the current status and affects future development. After the decision is determined at each stage, a decision sequence is formed, and thus the entire process is determined.
An activity route of the process. This kind of problem is regarded as a multi-stage process with a chain structure, which is called a multi-stage decision-making process. This problem is called a multi-stage decision-making optimization problem.

[Example 1] The shortest path problem. The figure shows a map. Each vertex in the map represents a city. The line between the two cities represents a road. The value on the line represents the length of the road. How can we get the shortest distance and the shortest length from City A to city e?

[Analysis] the entire process from A to E is divided into four stages, and K is used to represent the stage variables. in Stage 1, there is an initial State A and two optional branches (ABL and Ab2; stage 1 has two initial statuses: B1 and B2, and B1 has three available branches. B2 has two available branches ....... Use DK (XK
, XK + 1
) Indicates that XK is in the initial status at the K stage.
The initial status of the next stage XK + 1
Path Distance of, FK (XK
) Indicates the XK from the K stage.
The shortest distance from end to end e is solved by using the reverse push method. The specific calculation process is as follows:

S1: K = 4, with: F4 (D1) = 3, F4 (D2) = 4, F4 (D3) = 3

S2: K = 3, with: F3 (C1) = min {D3 (C1, D1) + F4 (D1), D3 (C1, D2) + F4 (D2 )} = min {8, 10} = 8

F3 (C2) = D3 (C2, D1) + F4 (D1) = 5 + 3 = 8

F3 (C3) = D3 (C3, D3) + F4 (D3) = 8 + 3 = 11

F3 (C4) = D3 (C4, D3) + F4 (D3) = 3 + 3 = 6

S3: K = 2, with: F2 (B1) = min {d2 (B1, C1) + F3 (C1), D2 (B1, C2) + F3 (C2 ), d2 (B1, C3) + F3 (C3)} = min {9, 12, 14} = 9

F2 (m) = min {d2 (B2, C2) + F3 (C2), D2 (B2, C4) + F3 (C4)} = min {16, 10} = 10

S4: K = 1, with: F1 (A) = min {D1 (A, B1) + F2 (B1), D1 (A, B2) + F2 (B2 )} = min {14,13} = 13

 

Therefore, the shortest path from point A to point E is a-> B2 1> C4-> D3-> E. The shortest distance is 13.

From the above process, we can see that in each stage, we can find the shortest path and shortest distance from each initial state of this stage to the end e of the process. When the reverse direction is pushed to process start, the shortest path and shortest distance of the entire process are obtained, and a set of optimal results (that is, the optimal results from each state of each stage to end e) are obtained ).

In the multi-stage decision-making problem in the above example, the decision adopted in each stage is generally related to time. The decision depends on the current state and immediately leads to state transfer, A decision sequence is generated in a changing state, so it has a "dynamic" meaning. This method is called dynamic planning to solve the problem of multi-stage decision optimization.

Basic concepts of the above case analysis and dynamic planning
The basic model of dynamic planning is as follows:

(1) determine the decision object of the problem.

(2) divide the decision-making process into stages.

(3) determine the status variables for each stage.

(4) determine the billing and target functions based on the status variables.

(5) establish the transition process of state variables in each stage and determine the state transition equation.

 

 

II,
Basic knowledge of Dynamic Planning

Dynamic Planning is a kind of optimization method. It is widely used in economics, engineering technology, enterprise management, industrial and agricultural production and military fields. In recent years, dynamic rules have been used in ACM/ICPC
It is not only common to draw (or partially apply dynamic planning thinking) questions, but also in a variety of forms. In almost all kinds of informatics competitions, applying dynamic planning to solve problems has become a trend, which is similar to dynamic planning.
Its advantages are irrelevant.

1. Common Terms for Dynamic Planning

Before learning about dynamic planning, you must first understand the following terms. This book simplifies standard terms for better understanding.

(1) status (smte)

For a problem, all possible conditions (including initial conditions and target conditions) are called a state of the problem.

(2) state variable (SK)

Associate a state variable Sk with each State K. Its value indicates the current solution value of the problem corresponding to state K.

(3) Decision)

Decision-making is an option. You can select a route or method for each status to reach the next status.

(4) decision variables (DK)

The value of the decision variable dk under status K indicates the current decision of status K.

(5) Policy

A policy is a set of decisions. When we solve a problem, we record a series of decisions as a policy. A policy that meets certain optimal conditions is called an optimal policy.

(6) state transfer function (t)

From one status to another, you can follow certain rules. We use a function t to describe such a rule, which maps state I and decision variable di to another State J, as T (I, DI) = J

(7) State Transition equation (f)

State transition equation f describes the mathematical relationship between state variables. In general, in response to the optimization problem, the state transition equation represents the conditions for Si Value optimization, or the formula for calculating the optimal value of the problem corresponding to state I, represented in algebra:

SI = f ({(SJ, DJ) | I = T (J, DJ), all feasible values for decision variable DJ })

2. Optimization Principle

In 1951, the American mathematician R. Bellman and others transformed the multi-stage decision-making problem into a series of interrelated single-stage problems based on the characteristics of a class of multi-stage problems, and then individually
Solution. Some static models can be converted into multi-stage dynamic models by artificially introducing the "time" factor and dividing them into time periods. They can be processed using dynamic planning methods. At the same time, he proposed to solve such problems
Principle
Of optimality ):

"The optimal decision of a process is of the following nature: regardless of its initial status and initial decision, in the future, the policies must form the optimal strategy for the process of taking the State formed by the first decision as the initial state ". In short, the sub-strategy of an optimal policy is also optimal for its initial and final states.

If this "Optimization Principle" is described in a mathematical language, it is assumed that in order to solve a certain optimization problem, N decisions must be made in sequence: D1, D2 ,..., DN. If the decision sequence is optimal, for any integer k, 1
<K <
N. No matter what the previous K decisions are, the future optimal decision depends only on the current State determined by the previous decision, that is, the future decision Dk + 1, Dk + 2 ,..., DN is also the best.

The optimization principle is the basis of dynamic planning. If you lose the support of this optimization principle, you cannot use the dynamic planning method for calculation.

3. What is dynamic planning?

Dynamic Planning is a branch of operational research. Dynamic Planning is more appropriate than a method of thinking. Because dynamic planning does not have a fixed framework, even if it is applied to the same channel
You can also create multiple forms of algorithms. Many algorithms on the implicit graph, such as the Dijkstra Algorithm for Finding the single-source shortest path and the breadth-first search algorithm, all penetrate the idea of dynamic planning. And
Many mathematical problems seem to be different from dynamic planning on the surface, but their solutions are completely consistent with dynamic planning.

Therefore, dynamic planning does not provide a set of models as deep or breadth-first. It can be used as needed. It must analyze and handle specific problems, it requires a wealth of imagination to build models and creative ideas to solve them.

4. What problems are dynamic planning suitable?

To be accurate, dynamic planning is not omnipotent. It is only suitable for solving optimal policy problems with certain conditions.

Perhaps, we may be disappointed to hear this conclusion: in fact, this conclusion does not reduce the brilliance of dynamic planning, because there are many problems in the above scope, there are also many problems that seem to be not in this scope that can be converted into such problems.

The "satisfying certain conditions" mentioned above mainly refers to the following two points:

(1) The State must satisfy the optimization principle;

(2) The status must be non-postpaid.

The so-called "no-effect" refers to: "past decisions can only influence future development through the current status. The current status is a summary of previous decisions ".

What does this feature mean? It indicates that dynamic planning is suitable for solving the problem that the current decision has nothing to do with the past status. Status, which appears at any position of the policy. It has the same status and can make the same decision. This is the meaning of no aftereffects.

5. Benefits of using dynamic planning to solve problems

After talking about so many dynamic plans, what benefits does it bring to our solutions?

In fact, the biggest advantage of dynamic planning is its high efficiency, and dynamic planning also has other advantages, such as: dynamic planning can produce a series of solutions, clear and simple algorithms, easy programming, easy adjustment, and so on.

 

 

 

Iii. Optimization Principle and no aftereffect

 

The basic components of the dynamic planning model have been introduced above. The problem to be solved now is: what kind of "multi-stage decision-making problem" can be solved using the dynamic planning method?

Generally, problems that can be solved using dynamic planning methods must meet the principle of optimization and the principle of no aftereffect.

 

(1) Optimization Principle of dynamic planning. The optimal strategy for the entire process has the following properties: regardless of the past status and decision-making, the remaining decisions must constitute the optimal strategy for the current State formed by the previous decision.

The local optimum that can be solved as a sub-problem in general geography will lead to the global optimum of the entire problem, that is, the problem has the optimal sub-structure, that is, the optimal solution of a problem depends only on the optimal of its sub-problems.
Non-optimal solutions have no impact on the solution. In example 1, in the shortest path problem, the path from any point on the optimal path from A to E must also be an optimal path from this point to end e, meeting the needs of optimization.
Principle. Next we will discuss another issue.

Example 2: path with the least remainder.

There are four points, namely A, B, C, and D. Two adjacent points are connected to c2k.
C2k-1
(1 ≤ k ≤ 3) represents two roads. The number on the Link indicates the length of the road. Defines the path from A to D with the minimum length divided by 4 to the optimal path.

Find an optimal path.

 

[Analysis] In this case, an error occurs if the problem is solved according to the method in Example 1. According to the idea in Example 1, the optimal value of A can be determined by the optimal value of B, while the optimal value of B is (1 + 3)
MoD 4 = 0, so the optimal value of A should be 2, but in fact, the optimal value of the path C1-C3-C5 can be (2 + 1 + 1) mod 4 =
0. Therefore, the optimal path of B is not the sub-path of A's optimal path. That is to say, the optimal value of A is not determined by the optimal value of B, that is, it does not meet the optimization principle, the problem does not have an optimal sub-structure.

It can be seen that not all "decision-making problems" can be solved by "Dynamic Planning", and the use of "Dynamic Planning" to deal with problems must meet the optimization principle.

 

(2) The principle of no aftereffect in dynamic planning. The principle of "No aftereffect" refers to the following nature: once the status of a stage is determined, the evolution of the subsequent process is no longer affected by previous states and decisions. That is
"The future has nothing to do with the past". The current state is a complete summary of previous history. The previous history can only influence the future evolution of the process through the current state. Specifically, if a problem is divided into different levels
After Section, phase
The status in I can only be set by stage I + 1
The State in is obtained through the state transition equation and has no relationship with other States, especially with the State that has not occurred. This is no aftereffect. From the perspective of graph theory, if we define the state in this problem
Vertices in the figure. The transition between two States is defined as edges, and the increment of the Weight Value in the transition process is defined as the edge weight, which forms a Directed Acyclic weighted graph. Therefore, this figure can be sorted by topology, at least
Stages are divided in the order of their topological order.

Let's take a look at the two examples below.

[Example 3] Question about the carrier. For the N points given by the plane, programming determines a closed travel route problem connecting each point. The figure shows the solution of the seven points.

[Example 4] travel route problems. On the basis of the carrier's question, if this travel route is specified to start from the leftmost, strictly from left to right to the rightmost point, and then strictly from right to left, find the shortest path length of the entire journey. The figure shows solutions to seven points.

Example 3: goods carrier

Example 4 Travel Roadmap

 

[Analysis] these two problems seem very similar, but they are completely different in nature. To facilitate the discussion, you can mark each vertex number. Because the path must pass through vertices 7 on the far right
(P1-P2) can be seen as the combination of two (P1-7) and (P2-7. Therefore, the state of this question can be expressed in the form of a combination of two roads. You can set the starting top of two paths in these statuses
The state of the same vertex is a stage, which is set to stage [p1, p2].

For travel route problems, if the phase [p1, p2] can be launched by phase [Q1, Q2], the following conditions must be met: PL
<Q1 or P2 <
Q2. For example, a road in stage [3, 4] can be obtained by adding an edge 4-5 to the road in stage [3, 5], but the state of stage [3, 5] cannot be obtained by the State in stage [3, 4, because
Travel routes must be strictly left-to-right. Therefore, if you already know the status in phase [3, 4], the status in phase [3, 5] must be known. Therefore, the problem cannot be solved.
The dynamic programming method can be considered.
 

There is no "sequence" between the phase and the phase for the cargo carrier issue ". For example, if the road {3-2-5--4-6-7} belongs to stage [3, 4], the road belongs to stage [2, 4] {2 -5--4-6-7}, while the road {2-3-6--4-5-7} belongs to stage [2, 4], it can be launched by {3-6-7, 4-5-7} of the phase [3, 4. If the vertex is used to represent the stage and the launching link is used to represent the edge, the relationship between phase [3, 4] and phase [2, 4] is shown on the right. We can clearly see that the relationship between these two stages is "ineffective" . Because there is a "loop" in this figure ". This problem cannot be solved as in the previous issue.

Phase relationship diagram,

 

 

 

 

Iv. General model of dynamic planning and design methods

 

The problem solved by dynamic planning is a multi-stage decision-making problem. Generally, it starts from the initial state and ends with the decision-making in the intermediate stage. These decisions form a decision sequence and determine the completion
An activity route of the entire process (usually the Optimal Activity route ).. The Design of dynamic planning has a certain pattern. The following steps are generally taken.

── ┐ ┌ ── ─ ┐ ──

Initial status → │ Decision 1 │ → │ Decision 2 │ →... →│ Decision N │ → end status

── ┘ └ ── ─ ┘ ──

Figure 1
Dynamic planning and decision-making process

(1) division stage: divide the problem into several stages based on the temporal or spatial characteristics of the problem. During the division phase, note that the division phase must be orderly or sortable. Otherwise, the problem cannot be solved.

(2) Determining the Status and status variables: express the objective situations in which the problem develops to various stages in different States. Of course, the selection of status must be ineffective.

(3) determine the decision and write the state transfer equation: because the decision and state transfer have a natural relationship, the state transfer is to export the state of this stage according to the status and decision of the previous stage. Therefore, if the decision is determined, the state transition equation can be written. But in fact, it is often done in turn, based on the relationship between the two adjacent States to determine the decision.

(4) Finding the boundary condition: the state transition equation given is a recursive formula that requires a recursive termination condition or boundary condition.

(5) Program Design implementation: the main difficulty of dynamic planning lies in the theoretical design. Once the design is completed, the implementation part will be very simple.

According to the above dynamic planning and design steps, the general solution framework 2 can be obtained.

Figure 2
General Model of Dynamic Planning and Design

The above provides a general model of dynamic planning. For simple dynamic planning problems, you can design dynamic planning step by step.

The following is a typical example of using the dynamic programming method.

[Example 6] digital triangle problem. Figure 3 shows a digital triangle pagoda. The number in a digital triangle is an integer not greater than 100. It is now required that you go from the top layer to the bottom layer, and each step can go down along the left or right diagonal lines.

Task 1: assume that the number of triangle rows is less than or equal to 10, input a definite integer m on the keyboard, and program to determine whether a path exists, so that the sum of the numbers passing along the path is exactly M, if yes, all paths are provided. If no, "no" is output.
Answer !" .

Task 2: assuming that the number of triangle rows is less than or equal to 100, the programming solution goes from the top layer to the bottom layer of a path, so that the total number passing along the path is the largest, and the maximum output is the maximum.

Input data: input data from the file. the first row of the file in Task 1 is the number of rows N and the integer of the triangle.
M. The next n rows are numbers in each layer from the top to the bottom. The file data format in Task 2 is the same as that in Task 1, but there is no integer m in the first row. In the example, the file data of Task 2 is shown as follows:

 

Input: 5

7

Output:

3 8
7


Output path and maximum value

8 1 0
38

Or "no answer !" .

2 7 7 4
810

4 5 2 6 5
2744

Figure 3 digital triangle 45265
 

[Analysis] It is easy to use enumeration to solve this problem, that is, to list all paths and record the total number of each path. Then determine whether the sum of the numbers is equal to the given integer m or
The idea of finding the largest sum of numbers is intuitive, and for Task 1, the number of triangles is not large because the number of rows is not large (<= 10, it should be implemented. However, for tasks
2. if we use the enumeration method, when the number of rows in a triangle is equal to 100, the enumeration amount is big. Obviously, the enumeration method is not suitable for solving Task 2. In fact, as long as you analyze Task 2 a little,
We can draw a conclusion:

If you get an optimal path from top to bottom, for each intermediate point in the path, the number passed by the path from the top to the intermediate point is also the largest. Therefore, this problem is a typical multi-stage decision optimization problem. Algorithm design and analysis are as follows:

For Task 1, the enumeration method can be properly confirmed to optimize the problem solution. There are only two ways to go from the top to the bottom, that is, left or right. Set "0" to indicate left,
"1" indicates the right, for a number tower with N layers, a walk from the top to the end can be expressed by a binary number of a N-1 bit. For example, if the binary number string is 1011, the corresponding path should be
Yes: 8-1-4-6. In this way, we can use an N-l bit binary number to simulate and determine the range of the solution. The path of the N-1 binary string corresponding to the decimal number ranging from 0 to 2n-1
The sum of the numbers in to determine whether it is equal to m and then obtain the solution to the problem.

For Task 2, use the forward solution in dynamic planning. Split the phase by triangle rows. If the number of rows is
N, the problem can be considered as a decision-making problem in n-1 stages. Starting from the starting point, find the first and second phases in a forward direction ...... The optimal path from the decision point to the start point in the n-1 stage. Finally, the optimal path from the start point to the end point is obtained.

Set: FK (UK) to have an optimal path from the vertex UK in the K stage to the vertex of the triangle. the total number that the path goes through is the largest, FK (UK) indicates the number and;

Since each decision has two options, either along the left slash down or along the right slash down, set:

Uk1 is a point in the K-1 stage UK along the left oblique line down;

Uk2 is the UK point down along the right diagonal line in the K-1 phase;

DK (uk1) is the number of uk1 in K stage, and DK (uk2) is the number of uk2 in K stage.

Therefore, a forward relation (state transition equation) can be written as follows:

FK (UK) = max {fk-1 (UK) + DK (uk1), fk-1 (UK) + DK (uk2)} (k =, 3 ,..., N)

F0 (U0) = 0

After a smooth push, you can find the N paths from top to bottom, and the maximum values are the correct answers to the N numbers passed by the N paths.

 

 

Http://apps.hi.baidu.com/share/detail/30927878

Http://blog.csdn.net/penzo/archive/2010/11/06/5992383.aspx