Dynamic Planning: from novice to expert

Preface

This article is translated from a topcoder article: Dynamic Programming: from novice to advanced. it is not strictly translated word by word. It includes some of your own understandings. The level is limited.

Preface _

A large part of the problems we encountered can be solved using dynamic planning (DP. Solving such problems can greatly improve your abilities and skills. I will try to help you understand how to use DP to solve problems. This article is based on examples, because the theory of ganbaba is hard to understand.

Note: Skip this section if you do not want to read it.

Introduction (Getting Started)

What is dynamic planning? How do we describe it?

A dynamic planning algorithm is generally based on a recursive formula and one or more initial states. The solution of the current subproblem will be introduced by the solution of the previous subproblem. Using Dynamic Planning to solve problems requires only polynomial time complexity. Therefore, it is much faster than backtracking and brute force.

Now let's take an example to understand the basic principles of DP.

First, we need to find the optimal solution for a certain state, and then find the optimal solution for the next state with its help.

What does "status" mean and how to find it?

The "status" is used to describe the subproblem of the problem. There are two paragraphs in the original text that the author does not know clearly. Skip the example above.

If we have several coins with a nominal value of 1 yuan, 3 yuan, and 5 yuan, how can we use a minimum coin to make up for 11 yuan? (On the surface, this question can be obtained using a greedy algorithm, but it cannot be solved by a greedy algorithm, for example, when the value of 1 yuan is changed to 2 yuan)

First, let's think about a question: how can we use the least coin to make up for I yuan (I <11 )? Why? Two reasons: 1. When we encounter a big problem, we are always used to narrowing down the problem so that we can analyze and discuss it easily. 2. the problems after the scale-down are the same as those of the original ones. Except for the scale-up, the other problems are the same, in essence, it is still the same problem (The problem after the scale becomes smaller is actually a sub-Problem of the original problem ).

Well, let's start with the smallest I. When I = 0, that is, how many coins do we need to make up for 0 yuan. Because, 5 are all greater than 0, that is, there is no smaller value than 0, so we need at least 0 coins to make up for 0 yuan. (This analysis is silly, isn't it? Don't worry. This idea helps us clarify what dynamic planning is doing .) At this time, we found a tag to indicate that "we need at least 0 coins to make up for 0 yuan ." It will be more convenient. If it is always expressed in plain text, you will feel like it will be around in a short time. Then, we use d (I) = J to show that I yuan needs at least J coins. Therefore, we have obtained D (0) = 0, indicating that 0 coins are required at least. When I = 1, only one coin with a nominal value of 1 is available. Therefore, we can pick up a coin with a nominal value of 1. Then we only need to make up 0 RMB, but this is already known, that is, D (0) = 0. Therefore, D (1) = D (1-1) + 1 = D (0) + 1 = 0 + 1 = 1. When I = 2, only the coin with a nominal value of 1 is available, so I picked up a coin with a nominal value of 1, next, I only need to make up 2-1 = 1 yuan (remember to use the minimum number of coins), and I already know the answer. So D (2) = D (2-1) + 1 = D (1) + 1 = 1 + 1 = 2. Until now, you may feel bored, just like you are a primary school student. Because we can only operate coins with a nominal value of 1 all the time! Be patient. Let's see what I = 3. When I = 3, there are two types of coins we can use: 1 yuan and 3 yuan (5 yuan is still useless, because the number you need to gather is 3 yuan! Too many RMB 5 ). Since there are two types of coins that can be used, I have two solutions. If I took a 1-dollar coin, my goal would be to: make up the minimum number of coins required for 3-1 = 2 yuan. That is, D (3) = D (3-1) + 1 = D (2) + 1 = 2 + 1 = 3. In this solution, I took 3 1 Yuan coins. In the second solution, I picked up a 3 Yuan coin and my goal was: make up the minimum number of coins required for 3-3 = 0 yuan. That is, D (3) = D (3-3) + 1 = D (0) + 1 = 0 + 1 = 1. this solution is about taking a 3-dollar coin. Okay. Which of the two solutions is better? Remember, we need to make up 3 yuan with the minimum number of coins. So how can I choose D (3) = 1? The specific result is as follows: D (3) = min {d (3-1) + 1, D (3-3) + 1 }.

OK. Let's take a look at the specific things with so many characters. From the above texts, we need to extract two very important concepts in Dynamic Planning: State and State Transition equations.

In the above text, d (I) indicates the minimum number of coins required by I yuan. We define it as the "status" of the problem. How can we find this status? I wrote in another article about the knapsack problem of Dynamic Planning (I): The status is defined according to the subproblem. When you find the sub-problem, the status will pop up. The final problem we want to solve can be expressed in this state: D (11), that is, the minimum number of coins required to make up for 11 yuan. What is the state transition equation? Since we use d (I) to represent the State, the state transition equation naturally contains d (I). The equation above containing State d (I) is: D (3) = min {d (3-1) + 1, D (3-3) + 1 }. Yes, it is the state transition equation, which describes how States are transferred. Of course, we need to abstract it,

D (I) = min {d (I-VJ) + 1}, where I-VJ> = 0, VJ indicates the nominal value of the J coin;

With the state and state transition equations, this problem is basically solved. Of course, talk is cheap, show me the code!

The pseudocode is as follows:

Is the solution when I ranges from 0 to 11:

It can be concluded that at least three coins are required to make up for 11 yuan.

In addition, by tracking how we are from the past status worth the current status value, we can find every coin that we use. For example, we can see from the figure above that the final result D (11) = D (10) + 1 (with a nominal value of 1), and D (10) = D (5) + 1 (nominal value 5), last d (5) = D (0) + 1 (nominal value 5 ). So the three coins we need at least 11 yuan are: 1 yuan, 5 yuan, and 5 yuan.

Note: There is still a section in the original article, but I have read it several times. It means that I have already analyzed it from I = 0 to I = 3. The author originally intended to talk about the plain text, but did not write the results, but it was not easy to understand, so this paragraph does not translate.

Elementary

A very simple example is discussed above. Now let's take a look at how to find the transfer mode between States (that is, find the state transfer equation) for more complex problems ). For this reason, we need to introduce a new word called recursive relationship to associate the State (the state transition equation is used)

OK. Let's see how it works.

A sequence has n numbers: A [1], a [2],…, A [n] to obtain the length of the longest non-descending subsequence. (LIS: longest increasing subsequence)

As we mentioned above, in the face of such a problem, we must first define a "state" to represent its subproblem and find its solution. Note: In most cases, a status is only related to the status that appears before it, but independent of the status that follows it.

Let's follow the idea of the simple question in the "getting started" section to find the "state" and "state transition equation" step by step ". Suppose we want to calculate a [1], a [2],…, The maximum length of the non-descending subsequence of a [I], where I <n, then the above problem becomes a subproblem of the original problem (The problem scale is reduced, you can let I = 1, 2, 3, etc. for analysis) Then we define d (I), which indicates the length of the longest non-descending subsequence ending with a [I] In the first I number. OK. You can estimate that d (I) is the status we are looking for against the simple question in "Getting started. If we calculate all the values from D (1) to D (n), the answer we are looking for is the largest one. The status is found, and the next step is to find the state transition equation.

To facilitate understanding how we find the state transition equation, I will first refer to the following example. If the sequence of the N numbers we require is:

5，3，4，8，6，7

Based on the status found above, we can get: (the longest non-descending subsequence below is represented by Lis)

• The LIS length of the first number D (1) = 1 (sequence: 5)
• The LIS length of the first two numbers D (2) = 1 (sequence: 3; 3 front is not smaller than 3)
• The LIS length of the first three numbers D (3) = 2 (sequence: 3, 4; 4 there is a 3 smaller than it, So D (3) = D (2) + 1)
• The length of LIS in the first four numbers D (4) = 3 (sequence: 3, 4, 8; 3 before 8 is smaller than it, So D (4) = max {D (1), D (2), D (3)} + 1 = 3)

OK, after analyzing this, I think the state transition equation is already obvious. If we have found D (1) to D (I-1), then d (I) the following State Transition equation can be used to obtain the result:

D (I) = max {1, D (j) + 1}, where j <I, a [J] <= A [I]

In the vernacular, to request d (I), add 1 to the length of the subsequence before I, where the last number is not greater than a [I, then the maximum length is d (I ). Of course, it is possible that the last number of subsequences before I is greater than a [I], then d (I) = 1, that is, it becomes a sub-sequence with a length of 1.

The analysis is complete. (the second column indicates the LIS length in the first I number, and the third column indicates the subscript In the LIS that reaches the previous number of the current number. Based on this, we can find the LIS sequence)

Talk is cheap, show me the code:

#include <iostream>using namespace std;int lis(int A[], int n){    int *d = new int[n];    int len = 1;    for(int i=0; i<n; ++i){        d[i] = 1;        for(int j=0; j<i; ++j)            if(A[j]<=A[i] && d[j]+1>d[i])                d[i] = d[j] + 1;        if(d[i]>len) len = d[i];    }    delete[] d;    return len;}int main(){    int A[] = {        5, 3, 4, 8, 6, 7    };    cout<<lis(A, 6)<<endl;    return 0;}

The time complexity of this algorithm is O (n2), which is not the optimal solution. There is also a clever algorithm that can reduce the time complexity to O (nlogn). There are various articles on the Internet that will not be repeated here. Portal: Lis O (nlogn) solution. You can also use "Sort + LCS" to solve this question. If you are interested, You can Google it on your own.

Exercise questions

An undirected graph G has n knots (1 <n <= 1000) and some edges. Each edge has a positive weight. Find the shortest path from node 1 to node N, or the output path does not exist.

Tip: in each step, for those nodes that have not been computed, and those nodes that have computed the shortest path from node 1 to it, if there is an edge between them, calculate the shortest path from node 1 to the uncomputed node.

Try to solve the following issues from the topcoder competition:

• Zigzag-2003 TCCC semifinals 3
• Badneighbors-2004 TCCC round 4
• Flowergarden-2004 TCCC Round 1

Intermediate

Next, let's take a look at how to solve the two-dimensional DP problem.

There are N * m grids on the plane, each of which contains a certain number of apples. Starting from the lattice in the upper left corner, you can only go down or to the right of each step. Every time you go to a grid, You will collect the apples in the grid, how many apples can you collect.

Solving this problem is almost the same as solving other DP problems. The first step is to find the "State" of the problem, the second step is to find the "state transition equation", and then basically solve the problem.

First, what is the "status" in this question? We must note that there are only two ways to get to a grid: From the left (except the first column) and from the top (except the first line ). Therefore, in order to find the maximum number of apples that can be collected after arriving at the current grid, we need to first check the grids that can reach the current grid and reach the maximum number of apples that can be collected. (Isn't it a bit difficult, but the essence of this sentence is actually the key to DP: To solve the problem, we must first solve the subproblem)

After the above analysis, it is easy to obtain the state and state transition equation of the problem. The State S [I] [J] indicates the maximum number of apples that can be collected when we go to the (I, j) grid. The state transition equation is as follows:

S[i][j]=A[i][j] + max(S[i-1][j], if i>0 ; S[i][j-1], if j>0)

I indicates the row, J indicates the column, and the subscript starts from 0. A [I] [J] indicates the number of apples in the lattice (I, j.

S [I] [J] has two calculation methods: 1. for each row, it is calculated from left to right and processed row by row from top to bottom; 2. for each column, it is calculated from top to bottom, and then processed from left to right one by one. The purpose of this is to calculate s [I] [J], s [I-1] [J] And s [I] [J-1] have been calculated.

The pseudocode is as follows:

The following two questions are from topcoder and used for practice.

• Avoidroads-2003 TCO semifinals 4
• Chessmetric-2003 TCCC round 4

Medium and high level

This section deals with the problem of DP with additional conditions.

The following is a good example.

An undirected graph G has n nodes, and its edge has a positive weight value.

You start from node 1 and start with m yuan. If you pass through node I, you will spend s [I] yuan (you can think of it as toll collection ). If you do not have enough money, you cannot pass through that node. Under such conditions, find the shortest path from node 1 to node n. Or the output path does not exist. If there are multiple shortest paths, the minimum cost is output. Limit: 1 <n <= 100; 0 <= m <= 100; for each I, 0 <= s [I] <= 100; as we can see, if there are no additional restrictions (You have to pay for the node, but you have not yet paid for the cost), then, this problem is the same as the classic dijela problem (finding the shortest path between two nodes ). In the classic dijela problem, we use a one-dimensional array to save the shortest path length from the Start Node to each node, M [I] indicates the length of the shortest path from the Start Node to the node I. However, in this case, we need to save the information about how much money we have left. Therefore, we naturally extend a one-dimensional array to a two-dimensional array. M [I] [J] indicates the shortest path length from the Start Node to node I, and the remaining J yuan. In this way, the problem is reduced to the original path to find the problem. In each step, for the shortest path that has been found, we find the next unmarked status (I, j) that it can reach ), mark it as accessed (and no longer visit this node), and find the path corresponding to the minimum value after adding the current Edge Weight Value in the shortest paths that can reach this node, the shortest path of the node. (We recommend that you draw a graph to understand it ). Repeat the preceding steps until all nodes are accessed. (The access here does not require us to go through it. For example, if a node has a high charge, you don't have enough money to go through it, but you have already visited it.) the minimum value in last min [N-1] [J] is the answer to the question (if there are multiple minimum values, if there are multiple shortest paths, select the path with the largest J, that is, the shortest path with the most remaining money ).

Pseudocode:

Below are several topics on topcoder for practice:

• Jewelry-2003 TCO online round 4
• Stripepainter-SRM 150 Div 1
• Quicksums-SRM 197 Div 2
• Shortpalindromes-SRM 165 Div 2

Advanced

The following questions need to be carefully explored before they can be resolved as available DP.

Problem: staradventure-SRM 208 Div 1:

Given a matrix of m rows and n columns (M * n grids), each grid contains a certain number of apples. You can only go down or to the right of the grid in the upper left corner. The destination is the grid in the lower right corner. Every time you walk through a grid, you collect the apples on the grid. Then, you can go back to the upper-left pane from the bottom right corner. You can only go to the left or up at a time. Similarly, you can collect the apples from the pane. Finally, once again, from the upper left corner to the lower right corner, you need to collect the apples in each grid (if the number of apples in the grid is 0, you do not need to collect them ). How many apples can you collect.

Note: When you pass through a grid, you need to take all the apples in the grid at one time.

Restrictions: 1 <n, m <= 50; the number of apples in each grid is 0 to 1000 (including 0 and 1000 ).

If we only need to go from the upper-left pane to the lower-right pane and collect the largest number of apples, the problem degrades to the one in the intermediate section. The problem here is reduced to a simple question in the "intermediate", which will be easier to solve. Let's analyze this problem. We can use DP only after the Protocol or modification. First, for the second path from the lower right corner to the upper left corner, we can regard it as the path from the upper left corner to the lower right corner, there is no difference. (That is, the optimal path from B to a is the same as the optimal path from A to B.) in this way, we get three paths that go from top to bottom. This can reduce the difficulty of the problem. Therefore, we can mark these three paths as left, center, and right paths. For Two Intersection paths (for example ):

Without affecting the results, we can regard them as two non-intersecting paths:

In this way, we will get three paths on the left, center, and right. In addition, if we want to obtain the optimal solution, the paths cannot be intersecting (except for the grids in the upper left corner and lower right corner ). Therefore, for each row of Y (except the first and last rows), the X coordinates of the three paths must meet the following requirements: X1 [y] <X2 [y] <X3 [Y]. After this step of analysis, the problem's DP Solution is further clarified. Let's consider line y, for every X1 [Y-1], X2 [Y-1] And X3 [Y-1], we have found a path that can collect up to the number of apples. Based on them, we can find the optimal solution for traveling y. Now we need to find the way to move from one row to the next. Order MAX [I] [J] [k] to indicate the maximum number of apples collected until the Y-1 line, where 3 paths stop in column I, J, K, respectively. For the next line of Y, add a grid (Y, I), (Y, J), and (Y, k) to each MAX [I] [J] [K) number of inner apples. Therefore, we move down each step. After moving this step, we also need to consider that a path may move to the right. (For each grid, we may move down from the top of it, or from the left to the right of it ). To ensure that the three paths do not overlap, we must first consider the path on the left to move to the right, then the middle, and finally the right path. For better understanding, let's consider the case where the path on the Left moves to the right. For every possible j, k pair (j <K), for each I (I <j ), consider moving from position (I-1, j, k) to position (I, j, k ). After processing the path on the left, process the intermediate path and then process the path on the right. Methods are similar.

Topcoder questions used for exercises:

• Minipaint-SRM 178 Div 1

Others

When reading a question and trying to solve it, first take a look at its limitations. If it is required to be solved within the polynomial time, the problem is likely to be solved using DP. In this case, the most important thing is to find the "state" and "state transition equation" of the problem ". (The State is not defined at will. Generally, after the State is defined, you need to find out how the current state is obtained from the previous state, that is, find the state transition equation.) If it looks like a DP problem, however, you cannot define the status, so try to normalize the problem to a known DP problem (as in the "advanced" section ).

Postscript

After reading this tutorial, we are far away from DP experts. coding is king.

Reproduced http://hawstein.com/posts/dp-novice-to-advanced.html