Dynamic planning: from novice to expert

Dynamic planning: From novice to expert March, 2013Hawstein
Source: http://hawstein.com/posts/dp-novice-to-advanced.html
Disclaimer: This document is licensed under the following protocols: Free reprint-Non-commercial-non-derivative-retention Attribution | Creative Commons by-nc-nd 3.0, reproduced please specify the author and source.

Objective

This article is translated from an article on TopCoder: Dynamic programming:from Novice to advanced, not strictly word-for-word translation, which joins some of their own understanding. The level is limited, but also hope to criticize.

Objective

A large part of the problems we encounter can be solved by dynamic programming (DP). Solving this kind of problem can greatly improve your abilities and skills, and I will try to help you understand how to use DP to solve problems. This article is based on an example, because the dry theory is very difficult to understand.

Note: If you already know something about one of these sections and don't want to read it, it's okay, just skip it.

Introduction (Getting Started)

What is dynamic planning and how do we describe it?

Dynamic programming algorithms are usually based on a recursive formula and one or more initial states. The solution of the current sub-problem is introduced by the solution of the last sub-problem. The use of dynamic programming to solve problems requires only polynomial time complexity, so it is much faster than backtracking, violence and so on.

Now let's take a look at the fundamentals of DP in an example.

First, we need to find the optimal solution of a state and then, with its help, find the optimal solution for the next state.

What does "status" mean and how do I find it?

The "state" is used to describe the solution of the problem's sub-problem. There are two paragraphs in the original that are not very clear, skip the direct example.

If we have several coins of $1, $3 and $5, how can we make up to $11 with the fewest coins? (The surface of this problem can be greedy algorithm, but greedy algorithm can not guarantee that the solution, such as 1 yuan to 2 yuan when)

First, we think of a problem, how to use the fewest coins to get enough I Yuan (i<11)? Why do you ask? Two reasons: 1. When we encounter a big problem, we always get used to making the problem smaller, so it's easy to analyze and discuss. 2. The problem of this size and the original problem is homogeneous, in addition to the size of the smaller, the other is the same, in essence it is still the same problem (the problem of the size of the problem is actually the sub-problem of the original issue).

OK, let's start with the smallest I. When the i=0, that is, how many coins we need to make up enough for 0 yuan. Since the 1,3,5 are more than 0, that is, there is no less than 0 of the value of the currency, so gather enough 0 yuan we need at least 0 coins. (This analysis is silly, isn't it?) Don't worry, this idea is good for us to clarify what the dynamic planning is doing. At this point we found a mark to express the phrase "enough for 0 dollars we need at least 0 coins." "will be more convenient, if you have been using plain text to express, in a moment you will feel very around." So, we use D (i) =j to show that I have enough I-yuan to need a minimum of J coins. So we've got D (0) = 0, which means it takes 0 coins to get the minimum of 0 dollars. When I=1, only a coin with a par value of $1 is available, so we pick up a coin with a par value of 1, then we just need to gather up to 0 yuan, and this is already know the answer, that is, d (0) = 0. So, d (1) =d (1-1) +1=d (0) +1=0+1=1. When i=2, still only a coin with a face value of 1 is available, so I picked up a 1-dollar coin, and then I just need to make up 2-1 = 1 yuan (remember to use the smallest number of coins), and this answer is already known. So D (2) =d (2-1) +1=d (1) +1=1+1=2. All the way to here, you may feel, so bored, feel like the subject of a pupil. Because we have been able to operate a coin with a par value of 1! Be patient, let's see what happens when we i=3. When the i=3, we can use two kinds of coins: 1 Yuan and 3 yuan (5 yuan is still useless, because you need to gather the number is 3 Yuan!). 5 Yuan too many pro). Since there are two kinds of coins that can be used, I have two options. If I took a 1-dollar coin, my goal would be: gather enough 3-1 = 2 dollars required for the minimum number of coins. i.e. d (3) =d (3-1) +1=d (2) +1=2+1=3. This plan says, I take 3 coins of 1 yuan; the second option is that I pick up a $3 coin, and my goal becomes: get enough 3-3 = 0 dollars for the minimum number of coins needed. i.e. d (3) =d (3-3) +1=d (0) +1=0+1=1. This plan says, I take 1 coins of 3 yuan. Well, which one of these two options is better? Remember we have to use the minimum number of coins to make up 3 yuan. So, choose D (3) = 1, how come? Specifically, this is achieved by: D (3) =min{d (3-1) +1, D (3-3) +1}.

OK, the code has so many words to say specific things, let's take a little abstract. From the above text, we need to draw two concepts that are very important in dynamic programming: State and state transition equations.

D (i) above indicates that the minimum number of coins needed for I-dollar is sufficient, and we define it as the "state" of the problem, how is this state found? I wrote in another article in the Dynamic Planning Backpack Question (a): Define the state according to the sub-problem. If you find a problem, the state will surface. Finally, the problem that we want to solve, can be expressed in this state: D (11), that is, to gather enough 11 yuan minimum number of coins required. What is the state transfer equation? Since we use D (i) to represent the state, then the state transition equation naturally contains d (i), the equation above which contains the state d (i) is: D (3) =min{d (3-1) +1, D (3-3) +1}. Yes, it is the state transition equation, which describes how states are transferred. Of course, we're going to abstract it,

D (i) =min{D (I-VJ) +1}, where I-vj >=0,vj represents the face value of the J-coin;

With the state and state transition equations, the problem is basically solved. Of course, talk is cheap,show me the code!

The pseudo code is as follows:

Is the solution when I from 0 to 11 o'clock:

It can be concluded that a minimum of 11 dollars is required for at least 3 coins.

In addition, by tracking how we are worth the current state value in a previous state, we can find out what denomination of coins we use each time. For example, from the above figure we can see that the final result D (one) =d (10) +1 (face value is 1), and D (Ten) =d (5) +1 (face value is 5), the last D (5) =d (0) +1 (face value is 5). So we have enough for $11 minimum 3 coins are: 1 yuan, 5 yuan, 5 yuan.

Note: There was a paragraph in the original, but I read it several times over and over again, probably meaning I have already shown in the above analysis from i=0 to i=3. The author originally wanted to speak some of the popular, the results did not write well, but more difficult to understand, so this paragraph does not translate.

Primary

A very simple example is discussed above. Now let's look at the more complex question of how to find the transition between States (that is, finding the state transition equation). So we're going to introduce a new word called a recursive relationship to link the state (or the state transition equation).

OK, on the example, see how it works.

A sequence has N number: a[1],a[2],..., a[n], the length of the longest non-descending sub-sequence is obtained. (Speaking of DP Basic will talk about a problem lis:longest increasing subsequence)

As we said above, in the face of such a problem, we must first define a "state" to represent its sub-problem, and find its solution. Note that in most cases, a state is related only to the state it appears in, and is independent of the later state.

Let's follow the idea of the simple question in the "Getting Started" section to find "state" and "state transition equations" one step at a way. If we consider the length of the longest non-descending subsequence of the a[1],a[2],..., a[i], where i<n, then the problem above becomes a sub-problem of the original problem (the problem size is reduced, you can let the i=1,2,3 and so on to analyze) then we define D (i), representing the number of previous I in A[i ] The length of the longest non-descending subsequence of the end. OK, against the simple question in "Getting started," you should be able to estimate that D (i) is the state we're looking for. If we calculate D (1) to D (N), then the answer we're looking for is the biggest one in the end. State is found, the next step is to find the state transition equation.

To facilitate understanding of how we find the state transition equation, let me first refer to the following example. If we ask for a sequence of this n number is:

5，3，4，8，6，7

According to the status found above, we can get: (the longest non-descending subsequence of the following is denoted by Lis)

• The first 1 numbers of LIS length D (1) =1 (sequence: 5)
• The first 2 number of LIS length D (2) =1 (sequence: 3;3 front no smaller than 3)
• The first 3 number of LIS length D (3) =2 (sequence: 3,4;4 Front has a smaller than it 3, so D (3) =d (2) +1)
• The first 4 number of LIS length D (4) =3 (sequence: 3,4,8;8 front is smaller than it has 3 numbers, so D (4) =max{d (1), D (2), D (3)}+1=3)

OK, analysis to this, I think the state transfer equation is already obvious, if we have found D (1) to D (i-1), then D (i) can be obtained by the following state transfer equation:

d(i) = max{1, d(j)+1},其中j<i,A[j]<=A[i]

The plain English explanation is that in order to require D (i), the sequence length of each subsequence in front of I, the last number not greater than a[i] is added 1, and the maximum length is d (i). Of course, it is possible that the last number in each subsequence in front of I is greater than a[i], then D (i) = 1, that is, it itself becomes a subsequence of length 1.

The analysis is finished,: (The second column indicates the length of the LIS in the number of previous I, the third column indicates that the index of the previous number of the current number in the LIS, according to which the LIS sequence can be obtained)

Talk is cheap, show me the code:

#include <iostream>UsingNamespaceStd;IntLis(IntA[],IntN){Int*D=NewInt[N];IntLen=1;For(IntI=0;I<N;++I){D[I]=1;For(IntJ=0;J<I;++J)If(A[J]<=A[I]&&D[J]+1>D[I])D[I]=D[J]+1;If(D[I]>Len)Len=D[I];}Delete[]D;Returnlen;} int main () {int a [] = {5 3486 7 }; cout<<lis (a 6) <<endl return 0;}        

The time complexity of the algorithm is O (N2), and is not the optimal solution. There is also a very ingenious algorithm can reduce the time complexity to O (NLOGN), the Internet has a variety of articles introduced it, here will not repeat. Portal: O (NLOGN) solution of LIS. This problem can also use "sort +lcs" to solve, interested in the words can be Google itself.

Exercises

The graph G has N nodes (1<n<=1000) and some edges with positive weight values on each edge. Find the shortest path to node 1 to node n, or the output does not exist for such a path.

Tip: In each step, for those nodes that have not been calculated, and those that have calculated from Node 1 to its shortest path, if there is an edge between them, the shortest path from node 1 to the non-computed node is computed.

Try to resolve the following issues from the TopCoder contest:

• ZigZag-2003 TCCC Semifinals 3
• BadNeighbors-2004 TCCC Round 4
• FlowerGarden-2004 TCCC Round 1

Intermediate

Next, let's look at how to solve the two-dimensional DP problem.

There are n*m on the plane, and a certain number of apples are placed in each lattice. You start with a grid in the upper-left corner, and each step can only go down or to the right, and every time you go to a grid, you collect the apples in the lattice so that you can collect as many apples as you like.

Solving this problem is hardly the same as solving other DP problems. The first step is to find the "state" of the problem, and the second step is to find the "state transition equation" and then basically solve the problem.

First, we need to find out what the "state" is in this question. One thing we have to notice is that there are at most two ways to get to a grid: from the left (except the first column) and from the top (except for the first row). So in order to find out the maximum number of apples that can be collected after reaching the current grid, we will first look at the squares that can reach the current lattice and reach the maximum number of apples they can collect. (is not a bit around, but the essence of this sentence is actually DP key: The solution of the problem of desire, first to find the solution of the sub-problem)

Through the above analysis, it is easy to obtain the state of the problem and the state transfer equation. State S[i][j] Indicates how many apples can be collected when we go to this lattice (I, j). Then, the state transition equation is as follows:

S[i][j]=A[i][j] + max(S[i-1][j], if i>0 ; S[i][j-1], if j>0)

Where I represents the row, J represents the column, subscript all starts from 0; A[i][j] represents the number of apples at the lattice (i, j).

S[I][J] There are two ways to calculate: 1. For each line, from left to right, and then from top to bottom row by line; 2. For each column, calculate from top to bottom and then column-wise from left to right. The purpose of this is to calculate s[i][j], s[i-1][j] and s[i][j-1] have been calculated.

The pseudo code is as follows:

The following two questions come from TopCoder, for practice.

• AvoidRoads-2003 TCO Semifinals 4
• ChessMetric-2003 TCCC Round 4

Senior

This sectionto discusses DP issues with additional conditions.

The following question is a good example.

The graph G has n nodes with a positive weight value on its edge.

You start from Node 1, and you start with M-dollar money. If you pass the knot I, then you will have to spend s[i] yuan (you can think of this as receiving tolls). If you don't have enough money, you can't go through that knot. Under such constraints, find the shortest path from node 1 to node n. or the output path does not exist. If there are multiple shortest paths, output the one with the least amount of money. Limitation: 1<n<=100; 0<=m<=100; For each i,0<=s[i]<=100; as we have seen, if there are no additional restrictions (charges at the junction, the cost is not yet given), then the problem is the same as the classic Dijkstra problem (find the shortest path between the two nodes). In the classic Dijkstra problem, we use a one-dimensional array to hold the length of the shortest path from the start node to each node, that is, M[i], which represents the length of the shortest path from the start node to the node I. In this issue, however, we have to keep the information about how much money we have left. So, naturally, we extend a one-dimensional array to a two-dimensional array. M[I][J] represents the shortest path length from the start node to the node I, and the remaining J-elements. In this way, we set this issue to the original path to find the problem. In each step, for the shortest path that has been found, we find the next unlabeled state it can reach (i,j), mark it as accessed (no longer accessing the node), and in each of the shortest paths that can reach the node, find the path that corresponds to the minimum value after the current edge weight value. is the shortest path to the node. (It's really a good idea to write a picture.) Keep repeating the above steps until all the nodes are accessible (the visit here is not asking us to pass it, such as having a node that is expensive, you don't have enough money to go through it, but you've already visited it) the minimum value in the last Min[n-1][j] is the answer to the question (if there are multiple minimums, That is, there are multiple shortest paths, so choose the largest path of J, that is, the shortest path that gives you the most money left.

Pseudo code:

Here are a few TopCoder questions for your practice:

• Jewelry-2003 TCO Online Round 4
• STRIPEPAINTER-SRM Div 1
• QUICKSUMS-SRM 197 Div 2
• SHORTPALINDROMES-SRM 165 Div 2

Senior

The following questions need to be carefully figured out to be a problem with the available DP solutions.

Question: STARADVENTURE-SRM 208 Div 1:

Given a matrix of M rows n columns (m*n), a certain number of apples are placed in each lattice. You start with a grid in the upper left corner and only go down or to the right, and the destination is the bottom right corner of the grid. When you walk through a grid, collect all the apples on the lattice. Then you go back to the upper-left corner of the grid from the lower right corner, you can only go to the left or up, the same, walk through a lattice and collect the apples inside. Finally, once again, you walk from the upper left to the bottom right, and every time you cross a grid, you also collect the apples (if the number of apples in the lattice is 0, you don't need to collect them). Please collect as many apples as you can.

Note: When you pass a lattice, you have to take the apples from the lattice at once.

Restrictions: 1 < N, M <= 50, the number of apples in each lattice is 0 to 1000 (contains 0 and 1000).

If we only need to go from the upper left to the grid at the bottom right and collect the largest number of apples, then the problem will degenerate into the "intermediate" section. The problem here is the "intermediate" of the simple question, which will be more good solution. Let us analyze this question, how to regulate or modify to use DP. First, for the second time from the lower right to the upper left corner of this path, we can see it from the upper left to the lower right corner of the path, there is no difference. (That is, the optimal path from B to a is the same as the optimal path from A to B), in this way we get three paths from the top to the end. An understanding of this can slightly reduce the difficulty of the problem. So, we can write these 3 paths as left, middle, and right. For two intersecting paths (for example):

Without affecting the results, we can treat them as two disjoint paths:

In this way, we will get the left, middle, and right 3 paths. In addition, if we want to get the optimal solution, the paths cannot intersect (except for the lattice that inevitably intersects the upper-left and lower-right corners). So for each row y (except for the first and last lines), the x-coordinate of the three path corresponds to: X1[y] < X2[y] < X3[y]. After this step analysis, the DP solution of the problem is further clear. Let's consider row y, for each x1[y-1],x2[y-1] and x3[y-1], we've found a path that collects up to the maximum number of apples. According to them, we can find the optimal solution of the travel Y. Now all we have to do is find a way to move from line to line. MAX[I][J][K] Indicates the maximum number of apples to be collected for the Y-1 Act, of which 3 paths are respectively listed in section i,j,k. For the next line y, the number of apples in the lattice (Y,i), (Y,j) and (y,k) are added to each max[i][j][k]. So, every step we move down. After we've made this move, consider that a path is likely to move to the right. (For each lattice, we may be moving down from it to it, or it may be moving from its left to the right to it). To ensure that 3 paths do not intersect, we first consider the left path to the right, then the middle, and finally the right path. For a better understanding, let's consider the case where the left path moves to the right, for each possible j,k pair (j<k), for each I (I<J), consider moving from position (i-1,j,k) to position (i,j,k). The left path is processed, the middle path is processed, and the path to the right is finally processed. methods are similar.

TopCoder Topics for Practice:

• MINIPAINT-SRM 178 Div 1

Other

When reading a topic and starting to try to solve it, first look at its limitations. If the requirement is resolved in polynomial time, then the problem is likely to be solved with DP. In this case, the most important thing is to find the "state" of the problem and the "state transition equation". (State is not a random definition, generally defined state, you want to find out how the current state is obtained from the previous state, that is, to find the state transition equation) if it appears to be a DP problem, but you can not define the state, then try to set the problem to a known DP problem (as in the "Advanced" section of the example).

Postscript

After reading this tutorial away from the DP experts are far from, good coding is the kingly way.

-

Random Posts

• 2014» "The Swift programming Language" Read thin
• Mar 2014» Read the book "Time as a friend"
• Jan 2014»google Java Programming style guide
• 2013» "Programming Zhu Ji Nanxiong" read thin
• Jul 2013» How to implement an LRU Cache with C + +

Dynamic planning: From Novice to expert (turn)