The way of arithmetic: metaphysical and predicate

One day in March 1966, Professor Andrew J. Viterbi of the University of California, Los Angeles, was explaining to graduate students The timing decoding algorithm SDCD of the winding code. But no matter how he explains it, students just don't understand. Reasoning, Viterbi that the students do not understand the reason is that the algorithm is too complex to prove. So he began to think about how to simplify the proof. After enduring irritability and confusion, he was inspired by the fact that the need to simplify is not the proof of the algorithm, but the algorithm itself. The SDCD algorithm was modified by Viterbi, and a probabilistic decoding algorithm based on Trellis was proposed. This algorithm is the cornerstone of the later famous CDMA technology. Viterbi also soared (founded Qualcomm, earning $ billions of).

A new algorithm brings revolutionary technology and the explosion of wealth, the role of the algorithm is not very small. But understanding the algorithm, changing life, or thinking of the algorithm to think, but for many people is the mirror flower, water months, difficult to touch.

Defined broadly, the algorithm is the Step (instruction) to solve the problem. Since the computer program is an instruction followed by an instruction (step) to proceed, it is actually the gradual expansion of the algorithm. Therefore, the algorithm is pervasive in all software programs, called the soul of the computer.

Understanding the soul is certainly not an easy thing to do. In addition to high abstraction, the logic behind the algorithm is very intertwined. After looking at an algorithmic solution to a problem, people feel not necessarily relaxed, but may be confused. How are these algorithms discovered or invented? How do the answers to these questions come to the idea of a particular algorithm? In some cases, it is not always possible to ask such a question, because the algorithm itself may not be understood. In fact, the deep mastery of the algorithm is not uncommon. Although many people will jiangshan in various occasions, the text, as if a master posture, but their understanding of the algorithm may be very superficial. Many people who often need to use the algorithm, but stay in the spontaneous, rather than the self-conscious stage: for the problems seen or similar to the problems seen in the question of how to answer, and for those who have seen the problem is not similar or less similar or the similarity is not easy to see the new problems are helpless.

The accumulation of content, boring statements, logical clutter and even understanding errors in the various algorithmic books aggravate people's fear of the algorithm.

The algorithm books on the market are dazzling, but there are common problems: to tell a lot of questions, list many algorithms, but fundamentally is the matter; there is no logical progression or hierarchical relationship between various algorithmic design or analysis strategies, and the order of the algorithm strategies is very arbitrary in arrangement, and does not conform to the causal association between them. For example, these books in the dynamic planning, static planning, greedy choice, approximation algorithm, etc., did not take into account the logical progression between the different strategies, just randomly arranged chapters, with some specific questions to explain these strategies. The result is that there is no logical main line through, reading laborious, scattered, can not form an organic whole. The only benefit of reading these books is to get answers to specific questions, but in the face of a new problem, it is unclear or even confusing how to use the design strategy and analysis strategy, or the order in which the strategies should be tried. In addition, these algorithms do not refine the algorithm strategy, but scattered in a variety of problems in the specific solution, the shape is not a height, scattered god more scattered, unable to systematically train the reader's algorithmic thinking.

These books can be consulted as reference books, but it is obvious that the reading or teaching materials as training algorithmic thinking have lost their strength.

So how to cultivate algorithmic thinking? The answer is the logic behind the algorithm. Different algorithmic strategies seem different, but in fact same strain, even from a higher level is the same kind of thinking. All algorithmic strategies such as divide-and-conquer, dynamic programming, greedy selection, randomization, approximation algorithms are just different aspects of the same thinking! There is a progressive relationship between them in logic and efficiency. Understand this point, the grasp of the algorithm will reach a new realm. We use the well-known minimum spanning tree problem as an example to illustrate this.

Figure 1 Minimum spanning tree problem, in which the rough lines form a minimal spanning tree

The minimum spanning tree problem is defined as follows.

The given input is: connected undirected graph with weights, each edge weighted.

Required output: A tree that connects all nodes, and is minimal.

For example, the thick lines in Figure 1 make up a minimal spanning tree of the graph.

How did we get the smallest spanning tree? Or the minimum spanning tree problem, what method should be used to solve it?

The simplest approach is of course a brute force strategy, where all spanning trees are found, their weights calculated, and the smallest trees can be taken out. But the cost of such a strategy is high, its order of magnitude, which represents the number of edges in the graph, representing the number of nodes in the graph. And it's a very difficult factorial level to get excited about. Obviously, we need to improve the algorithm. So how to improve it?

In the face of complex problems, humans often choose to simplify the problem and break down complex big problems into simple small problems. After solving the small problem, the solution of the small problem is merged into the solution of the big problem. This is called "Divide and Conquer". Because small problems are easier to solve than big problems, divide and conquer become the best policy, and evolve into the basic strategy of algorithmic design. For the minimum spanning tree problem, the entire graph is decomposed into two (or the other number) of equal or similar sub-graphs of the size (number of nodes), respectively, on these two sub-plots to find the smallest spanning tree, and then the search out of the two smallest spanning tree merged together. Obviously, the cost of decomposition is linear, but the cost of merging is, so that we get the cost of the split algorithm recursively. According to the master solution, the solution of the recursive formula is.

Is this the optimal solution? A careful analysis of the cost component of the above-mentioned divide-and-conquer algorithm reveals that there are many duplicate sub-sub-problems in the decomposition of sub-problems, and it is obviously not advisable to solve these same sub-problems repeatedly. The improvement is to resolve the duplicate sub-problems once and then save the results for later use, which is the dynamic planning strategy. With such a strategy, the most conservative can also reduce costs to (in fact, reduce costs to a slight optimization).

But is this the optimal solution? Actually, not yet. Careful analysis can be found, because the building is the smallest spanning tree, we can select the smallest side of the graph as the smallest tree edge, the condition is that the newly selected edge does not have the selected edge to form a loop until a side is selected. And this is the greedy choice strategy. Because of the time complexity of sorting all the edges, it is linear to check if an edge is selected to form a loop with the previously selected edge, so the time cost of the entire algorithm is. If, the efficiency of this algorithm will be higher than the efficiency of the Standard division strategy discussed earlier. If an improved data structure is used to support a greedy selection strategy, the time complexity can be reduced to (using the amortization time of the Fibonacci Polachi).

But the greedy choice strategy is not the optimal strategy yet. By carefully analyzing the construction algorithm of the minimum spanning tree above, we find that its cost lies in the choice of edges (sorting is prepared for selection), while the cost of constructing the smallest spanning tree itself is only. Why do we have to spend more than the cost of structuring ourselves to construct the smallest spanning tree? If we know which side belongs to the smallest spanning tree, then it is easy to construct it. But to know which edges belong to the smallest spanning tree, does it need to be compared with other edges? Maybe we don't need it. We can use a random number to tell us whether an edge belongs to the smallest spanning tree. To be precise, we use a coin toss to determine whether an edge should be included in the minimum spanning tree. This reduces the time cost to linear. This is the strategy of randomization.

Table 1 The minimum spanning tree construction cost in the strategy progression of the algorithm

Thus, as the algorithmic strategy progresses, the cost of finding the smallest spanning tree is decreasing, and the stochastic strategy achieves a linear minimum (table 1)!

However, careful readers may have many problems. For example, under the greedy choice strategy, why would you want to use Fibonacci Polachi? How is the amortization analysis of Fibonacci Polachi derived? Under the randomization strategy, how do you ensure that the selected edges are really the edges of the smallest spanning tree? In addition, the greedy selection strategy in table 1 seems to be comparable to a randomization strategy: and not an order of magnitude (usually greater than)? How to say with the progressive of the algorithm strategy, the cost is continuously reduced?

Under the greedy selection strategy, each time we select the least weighted edge to join a minimum spanning tree that is initially empty, the joined edges cannot form loops with the joined edges. Under this strategy, the maximum cost of the algorithm is on the smallest edge of each selection. The heap is the best data structure to support this kind of operation. But for a normal two-fork heap, each time you delete a heap top (we use the smallest heap of course), we need to adjust the heap to preserve the heap's properties, laying the groundwork for subsequent minimum-edge operations. Because the time for this adjustment is a number of levels, the operating cost of the entire minimum spanning tree is. But if we change our minds and cancel the two-fork heap, which requires no more than 2 of each node's limit, the operational cost of the adjustment heap will be reduced to the constant level. In this case, each time the heap top element is deleted, the larger sub-heap of the element is hung directly under the sub-heap of the element's smaller size. In this way, the total cost of the select edge operation for the minimum spanning tree appears to be. Plus the cost of the drop-off operation, the cost of the entire minimum spanning tree is.

Wait, this seems to have a problem: the heap adjustment operations described previously are constant, as long as the number of sub-heaps that appear after the top element of the heap is deleted. Otherwise, it may not be a constant time to select the smallest element in the middle of a heap of sub-heaps (thus hanging other sub-heaps below it). Therefore, the crux of the problem is to ensure that the number of sub-heaps generated by removing the top elements of the heap is constant or limited. This thought led to the advent of Fibonacci Polachi. In fact, the Polachi is further than this: the merge operation is not done immediately, but instead leaves the sub-heap, which merges when the number of sub-heaps exceeds a certain limit. In addition, a merge operation occurs when a heap with the same number of degrees is present (the top element of the heap has the same child nodes): A heap of the same degree is merged into a new heap. Thus, we can draw the following definition of Fibonacci Polachi.

The Fibonacci Polachi consists of a common set of heaps (non-binary heaps).

The maximum size of any subtrees tree of a node of K is (the number of child nodes per node cannot be exceeded).

All sub-heaps have different degrees.

Figure 2 shows a Fibonacci Polachi.

Figure 2 Fibonacci Polachi Structure

Under Fibonacci Polachi, the number of sub-heaps generated by deleting a heap top does not exceed. Therefore, the time cost of finding the smallest element within the remaining sub-heap will not exceed. But that doesn't seem to be the time we want. However, careful analysis found that under the limitations of the Fibonacci Polachi, it is not possible to have the number of child nodes per node. In fact, the majority of nodes have very few child nodes. Such a small number of nodes of the height of the resulting high cost can be diluted to a large number of low-level nodes in the lower operating costs, thereby reducing the operation cost of the entire minimum spanning tree to the boundary of the constant cost of each edge operation. This is the central idea of the amortization analysis.

For the minimum spanning tree algorithm of stochastic strategy, the edge must belong to the smallest spanning tree because the edge selection is random. Therefore, we also need some kind of testing when we randomly select edges to measure the suitability of this side. For this measure, we need to have some sort of edge, but this division and testing itself must be at a linear level. How do we do that? In view of space constraints, interested readers see the path of the algorithm: never to infinity.

For the order of magnitude and comparison, in the case of dense graphs, they are indeed an order of magnitude. But if the figure is sparse or and is an order of magnitude, then it boils down to that. If the number of nodes is large, it will be significantly higher. Moreover, the time cost of the greedy choice strategy is in the case of using complex Fibonacci Polachi, and it is amortization time! Therefore, in many aspects, the strategy of randomization is better than the greedy choice strategy.

As can be seen from this paper, the repeated deliberation of the minimum spanning tree problem could be used to string the whole design strategy of the algorithm. When these strategies are strung out, what we see is not only an individual strategy, but an algorithm that is not normally seen! Although looming, but for the eye, it does exist. This is the algorithm realm of "metaphysical and predicate".

The way of arithmetic: metaphysical and predicate