"An Introduction to Algorithms" Dynamic programming "optimal binary search Tree"

The previous two articles on the dynamic planning of "steel pipe cutting" and "matrix chain multiplication", feel this article, can also be regarded as the end of the work. In fact, according to the first two articles, to here, can also carry out some summary, we can find some regularity of things.

The so-called dynamic programming, in fact, is to solve recursive calls, there may be repeated calculation sub-problem, resulting in a lot of time to do the problem of repetitive work. The solution is to repeat the results of the sub-problem, first saved up, and so on, and then need to use the time, directly to use, and do not need to calculate.

But here are some things to keep in mind:

① to solve the problem, such as "pipe cutting" in the long steel pipe, its optimal solution, to be able to be decomposed into sub-problems of the optimal solution of the combination.

② How to give the sub-problem, grandson problem, such as the optimal solution of the unified recursive call form.

③ find one or several appropriate sets to preserve the solution to the sub-problem we are seeking, or to divide it. For example, in the pipe cutting, we use an array, and to the matrix chain multiplication, we use a two-dimensional array.

The ④ uses a top-down or bottom-up scheme.

I. Overview

Well, let's take a look at the optimal binary search tree. For binary search tree, you can refer to the previous two articles: two fork tree pre-and post-sequential non-recursive traversal implementation and two-fork search tree insertion and deletion . Don't repeat it here.

If we want to design a program, to achieve the translation of English text to French. We can set up a binary search tree, n English error as the keyword, the corresponding French word as the associated data. We then go through the English words of each node in the binary search tree in turn to find the right words for translation work. Here we certainly hope that the less time this search, the better, if only to consider the simple time complexity, we can consider the use of red and black trees, etc., to achieve the search time complexity of OLG (n). But for English words, the probability of each word appearing is different, such as "the" and other words. It is clear that the closer the word is to the root node the less time it takes to search. And for some very rare words, you can keep them away from the root node.

The question comes, if we know n different keywords ki and our probability of occurrence p[i], how can we make such an optimal binary search tree ?

Find this binary tree results are two, one is to find the key value we want, the other is to traverse the whole tree did not find our key value, we will not find the key value also arranged some node di to represent, then obviously the di node is a leaf node. Called di as pseudo-keyword, the corresponding probability is q[i]. The D0 represents all values less than K1, the DN represents all values greater than KN, and the Other Di (1<=i<=n-1), which represents the Di value between Ki and ki+1. Because a search either searches for di or searches for a ki, all of them are:

And now we can give a look at the number of nodes that will be searched once, because for a binary tree, each layer will find more than one node, so we can add 1 to the depth of each node, then multiply the node's pi, and then sum the results of all the nodes, You can find an expectation to search for a node that the tree will search for:

The definition of the optimal binary search tree is: For a given node key value probability, the two-fork search tree structure with the minimum expectation is made.

second, using dynamic programming to process the optimal binary search tree

So that's the steps:

① first divides the problem into sub-problems, to see whether the optimal problem is the sub-problem is also the best time can be achieved.

You can use the "jianqie-paste" method to prove that for an optimal binary search tree T, for a subtree of T ti, if it contains the keyword ki....kj, if ti is not the optimal binary search tree, then it means that there is another two-fork tree TJ containing the node ki...kj, So that TJ's search expectation is lower than TI's search expectations, then we can "cut" ti from the T, and then the TJ "paste" to the original position of TI, so that the new generation of trees will certainly be lower than the original tree T search expectations, which is contradictory, Because we originally assumed that T was an optimal binary search tree. So we can prove that the sub-tree of the optimal binary search tree is the optimal binary search tree.

So we can determine that the optimal binary search tree problem, can be decomposed into sub-problems, but also the sub-problem is the optimal binary search tree problem, and can easily meet, in the sub-problem, there will be a number of repeated sub-problems, Sun Tzu problems, so to meet the conditions of dynamic planning, so can use dynamic programming method to solve.

② The second problem is the unified form of the optimal solution of recursive sub-problems (computational formula)

At this time and with the previous "light pipe cutting" and matrix chain multiplication somewhat similar, it is easy to see, we will be a tree search expectation problem, decomposed into: Zuozi search expectation problem + root node search expectation + right subtree search expectation; For the root node, it has a depth of 0, so it is used directly (0+1) *p[ The root position], which is the search expectation. And for the left and right sub-trees, if they are a separate tree, then you can also directly use (the left subtree of the search expectation problem + the root node of the search expectation + right subtree search expectation) Form to recursion, but here, the tree is more than the simple two tree of the case of a layer of depth, because they have a more root node above the head, So it is necessary to add each node with a probability of their own occurrence p[i] or Q[i] (this includes the key byte points and pseudo-key byte points), so the increase is the sum of the probabilities of all nodes in each tree. So W (i,j) represents the tree containing the node ki...kj, the sum of the probabilities of all nodes, thus:

If we take the node in R as the root junction, we have:

And according to the above analysis, we will search for a tree, assuming that the root node at the KR, the probability of decomposition into root node p[r] and the left subtree of the search expected Kazo node probability and: E (i,r-1) +w (i,r-1), the right subtree of the search expectation plus the right subtree node probability and: E (r+1,j) +w (R+1,J).

One of the things to consider here is E (i,j), when J=i-1, when the subtree cannot contain the keyword, because the keyword ordinal i<=k<=j, and j=i-1, is obviously not satisfied. So the subtree will only contain a pseudo-key byte point: Di-1.

So according to the above two formulas, there are:

So we can find the recursive unified form of the sub-problem we need.

③ uses the appropriate data structure to store sub-problem processing results and partition information.

The data we need to save here are: The best search expectation E (i,j) for each subtree, the root node position R (i,j) at the best time of each subtree, and the probability and value of each subtree: w (i,j).

④ uses a bottom-up approach to solving sub-problems and parent issues.

So, according to the previous "matrix chain multiplication" idea, it is easy to give the following code:

const int maxval = 0x7fffffff;const int n = 5;//search to root node and virtual key probability double p[n + 1] = {-1, 0.15, 0.1, 0.05, 0.1, 0.2};d ouble q[ n + 1] = {0.05, 0.1, 0.05, 0.05, 0.05, 0.1};int root[n + 1][n + 1]; Record root node double w[n + 2][n + 2]; Sub-tree probability sum double e[n + 2][n + 2]; Subtree Expected cost */* p is an array that holds the probability of a critical byte point [1,n],q is an array of probabilities for storing pseudo-key byte points (leaf nodes) [0,n] * n is the number of nodes * E[i,j] is the expected cost of holding a search for the best binary book containing the keyword KI~KJ, due to include E [N+1,n] q[n] and e[1,0]q[0], so the range is [0,n+1] * w[i,j] is the probability of storing ki~kj and, W[i,j]=w[i,r-1]+p[r]+w[r+1,j] * Root[i,j] is the storage contains ki~ KJ keyword subtree, under optimal circumstances, the root node subscript */void optimal_bst (double* p, double* q, int n) {//First processing w[i,j] and e[i,j] i=j+1, this is the case q[i-1] for (int i = 1; I <= n + 1; i++) {w[i][i-1] = q[i-1];w[i][i-1] = q[i-1];} Then process the length l from 1 to N of the loop int l = 0; Represents the length of the ki~kj int j = 0; The subscript value representing the last element jint i = 0; The subscript value of an element that represents the starting iint r = 0; Represents the root node of the subscript double tmp = 0; Here to store the temporary result of the e array element calculation, so is also a double type for (L = 1; l <= N; l++) {//With I for Outer loop, here because the length is l,i maximum for n-l+1, and j-i+1=l,j=l+i-1for (i = 1; i lt;= n-l + 1; i++) {j = l + i-1;//first initialize E[I][J] and w[i][j]e[i][j] = MAXVAL;W[I][J] = W[i][j-1] + p[j] + q[j];for (r = i; r <= J; r++) {//formula: E[i][j] = e[i][r-1] + e[r + 1][j] + w[i][j]; TMP = E[i][r-1] + e[r + 1][j] + w[i][j];if (tmp < E[I][J]) {e[i][j] = tmp;root[i][j] = r;}}}}

The time complexity of the above algorithm is O (n^3), we can make a little optimization, let it become O (n^2), here need to use a conclusion:

The OPTIMAL_BST function is changed into the following form, changing the way the root node position r is traversed, and, of course, adding a judgment condition before:

/* * Optimized version */void Optimal_bst2 (double* p, double* q, int n) {//First handling w[i,j] and E[i,j] i=j+1, this case is q[i-1]for (int i = 1; i < = n + 1; i++) {W[i][i-1] = q[i-1];w[i][i-1] = q[i-1];} Then process the length l from 1 to N of the loop int l = 0; Represents the length of the ki~kj int j = 0; The subscript value representing the last element jint i = 0; The subscript value of an element that represents the starting iint r = 0; Represents the root node of the subscript double tmp = 0; Here to store the temporary result of the e array element calculation, so is also a double type for (L = 1; l <= N; l++) {//With I for Outer loop, here because the length is l,i maximum for n-l+1, and j-i+1=l,j=l+i-1for (i = 1; i lt;= n-l + 1; i++) {j = l + i-1;e[i][j] = maxval;w[i][j] = w[i][j-1] + p[j] + q[j];if (i = = j) {Root[i][j] = i;e[i][j] = p[i] + q[i -1] + q[j];}  else {//First initialize E[I][J] and w[i][j]for (r = root[i][j-1]; r <= root[i + 1][j]; r++) {//formula: E[i][j] = e[i][r-1] + e[r + 1][j] + w[i][j];tmp = e[i][r-1] + e[r + 1][j] + w[i][j];if (tmp < E[I][J]) {e[i][j] = tmp;root[i][j] = r;}}}}}

after the above modification, the time complexity of O (n^2) can be achieved.

Exercise 15.1-1, the function that prints the relationship of each node of the above optimal binary search tree to its parent node:

/* * Print optimal binary search tree */void print_optimal_bst (int i, int j, int r, bool root_flag) {int root_node = root[i][j];if (root_flag) {C Out << "root node: K" << root_node << endl;root_flag = False;print_optimal_bst (i, root_node-1, Root_node, RO Ot_flag);p Rint_optimal_bst (Root_node + 1, J, Root_node, Root_flag); return;} if (i > J + 1) {return;} else if (i = = j + 1) {if (J < R) {cout << "D" << J << "is" << "K" &L t;< R << "left child" << Endl;} else {cout << "D" << J << "yes" << "K" << R << "right Child" << Endl;} Return Add a return here because, when the loop to the i==j+1, the end, no longer recursive, there is no reasonable root_node} else {if (Root_node < R) {cout << "K" << Root_node << "yes" << "K" << R << "left child" << Endl;} else {cout << "K" << root_node << "yes" << "K" << R << "right Child" << Endl;} Print_optimal_bst (i, Root_node-1, Root_node, Root_flag);p Rint_optimal_bst (Root_node + 1, J, RooT_node, Root_flag);} 

Output Result:

"An Introduction to Algorithms" Dynamic programming "optimal binary search Tree"