Problem Complexity and Adversarial Lower &

Liu zitao

We usually talk about an algorithm, and we often care about its time complexity. Of course, we usually care about the time complexity upper bound, that is, the upper bound. Upper bound tells us about the performance of our algorithms in some bad cases. For example, our comparison-based sorting algorithm knows that mergesort (Merge Sorting) is O (nlgn), but we should be able to raise such questions: can we do it better? (Can we do better ?) Therefore, in this article, we will briefly introduce the lower bound of the algorithm, lower bound, which tells us whether there is a better algorithm for a problem.Lower bound tells us what cannot be done.

"Lower bounds tell us what cannot be done, they are very important in guiding us in our search for what can be done."

(Of course, I am google for lower bound, and there is very little Chinese information. I know that there are almost no lower bound involved in algorithm courses in colleges and universities, however, google's keywords above show that almost all colleges and universities in North America have relevant knowledge chapters, both undergraduate and graduate students. Is that the difference in education ??!! Here, the right is a getting started tutorial)

Regression topic:

Of course, we know that the so-called lower bound is to find Omega (n. (If you are not clear about shoes, you can check the link given at the beginning of the article)

I decided to give an example.

For example, we know that the comparison-based sorting algorithm is better than nlgn, so we should be able to prove that sorting = Omega (nlgn), of course, sorting refers to any sorting algorithm. In this case, we will find that the problem has come out. We need to prove that any algorithm meets a condition, arbitrary, arbitrary .... for some traditional examples, internal analysis of an algorithm is useless.

There are usually two methods for finding lower bound for the class of algorithms that solve a problem,

1) demo-treeWe can analyze the decision tree method to get the lower bound (this is also the method mentioned in the introduction to algorithms)

2) adversary strategyThis is really hard to translate, competitor strategy ?! Of course, this method is more tight than the lower bound obtained from the preceding demo-tree. (Of course, this is not absolute, but also depends on your strategy)

Basically, lower bound is introduced through the algorithm shared by sorting. I am not a mainstream here.

Next, we will use the lower bound example of sorting to explain the two methods mentioned above, and then give some examples to help you better understand them. Well. That's it.

Lets get started ~

For comparison-based sorting, we know that the sorting here is based on comparison! (This is a big truth, and it is also a nonsense O ~)

Now let's assume that the data to be compared is a1, a2 ,..., an, we know that for the n data (assuming there are no equal elements in it), we can make a full arrangement of the n data, which contains n! Type. So what is a demo-tree? I took a picture of an introduction to algorithms.

I cut such a large paragraph of text, because it is very detailed, and I hope that I will read it carefully. The number of each non-leaf node can be considered as the subscript I of the element ai to be compared. root indicates comparing a1 and a2. If it is less than walking left branch, it is greater than walking right branch. The following is similar. For a leaf node, the final result is a full arrangement of the sub-targets of the element. Of course, this is a possible final sorting result. This should be easy to understand. If it is hard to understand, I will give a comment, for example, the input is a1, a2, a3, a4, my final leaf node is [2 4 1 3], which indicates that the final sorting result is a2, a4, a1, a3.

This tree looks very simple. Of course, it still gives me a lot of useful information. First, we know that the complexity of the comparison algorithm is closely related to the number of comparisons. We can see from the tree that the number of comparisons from root to leaf along the road is the height of our branch. Of course, it is obvious that, we need to consider the maximum height of the tree, that is, the longest one. It indicates the number of comparisons we need in the worst case (here it seems that there is a conflict with the offline definition of the height of the tree, it is good to understand the meaning of O (distance _ distance) O ~)

Okay. Now we are looking for lower bound ~ Obviously, our tree is a binary tree, which is greater than left and less than right. Our final height is h. We know that there are at most 2 ^ h leaves with a full binary tree height of h, at the same time, we also have n data records, up to n! So according to our demo-tree, We can get:

N! <= 2 ^ h

Of course, this is a simple deformation. h> = lg (n !)

So h = Omega (nlgn ). (For children's shoes with unknown truth, please refer to the link given at the beginning, which contains some basic mathematical formulas)

What does this mean? This shows that for a comparison-based sorting algorithm, we need at least Omega (nlgn) comparisons, otherwise we will not be able to get the results of the bottom leaf node. Of course, the most important thing is that we have come to the conclusion that any sorting algorithm is random!

Next let's take a look at how to use Adversary Strategy to get this lower bound.

First, explain the idea of Adversary Strategy:

We can imagine that there is an Adversary. Of course we should call it God in Chinese ~ There is such a God, when we are comparing, we cannot get the comparison result immediately, we want to ask God, only God knows, (the power of God is not good .... ran .... back to the topic), whenever we want to compare, we need to get a comparison result from God.

Of course, for God, it always wants to delay us and ask as much as possible about the comparison results. So we have a direct adversarial relationship with God.

For sorting, we know a total of n! Such as a1, a2, a3, our possible results include [1 2 3] [1 3 2] [2 1 3] [2 3 1] [3 1 2] [3 2 1], let's call it the candidate set S. Every time we get the comparison results of two data from God, we can remove several elements from the candidate set, reduce the size of the candidate set | S |, when | S | = 1, our task is completed. What does God do? What God wants to do is to tell us a result every time, but it always tries its best to keep as many elements as possible in our candidate set. It is not hard to see that every time we get the remaining size greater than or equal to the previous average, that is, every time we get an answer from God, we can cut down at most the items in the candidate set. So to make | S | from n! To 1, we must at least ask lg (n !) So lower bound is Omega (nlgn ). Of course, if we didn't ask nlgn, our candidate set should contain at least two elements. At this time, the algorithm will give a result, and God will always say that the answer is different, in this way, our algorithm is fail, which means that none of the algorithms smaller than nlgn will work. (A bit similar to counterevidence)

The above demonstrates the lower bound of sorting in the worst case. Below we can also prove that in the case of average case, the lower bound of sorting is also Omega (nlgn ).

In fact, through the demo-tree method above, we already know that finding the lower bound under average case is actually finding the average lower bound of all the branches of the tree.

If our tree is a balanced binary tree, we know that every branch is nlgn, so the lower bound in average case is Omega (nlgn ), but isn't the comparison result of the demo-tree balanced binary tree? In fact, we only need to proveThe one that minimizes their average depth is a completely balanced tree.Obviously, if it is not a balanced binary tree, it means that the height difference between the deepest node and the shortest node is greater than or equal to two, at this time, we can move the deepest two nodes to the shortest pole, which reduces the average heights, but does not affect anything else. This shows that only for any unbalanced tree, we can all make his average height smaller, so it means that the average depth of leaves must be at least nlgn. Pass.

By the way, it can be proved that for all randomized sorting algorithm, the lower bound of the number of comparisons is also nlgn.Http://en.wikipedia.org/wiki/Randomized_algorithm

I decided to put this in the next article. O (distinct _ distinct) O ~

The following are some examples to illustrate the two methods mentioned above.

In this example, there are two sorted lists with a length of n, a1 <a2 <... <an, b1 <b2 <... <bn, We need to merge the two lists into a list to ensure that the list is increasing. We need to prove that the number of lower bound in the merge process is 2n-1.

Suppose we use adversarial strategy to prove that we only compared 2 N-2 times, that is, we only want to ask 2 N-2 times, then we know that there is an element ai, it must not be compared with bj or bj + 1. Assume that our ai is 2i-1, that is, all odd numbers, bi = 2i, and all even numbers. At this time, our algorithm has provided output, but for the omnipotent god, it knows that you are not comparing ai and bj, or are not comparing ai and bj + 1.

If you do not compare ai and bi, you output aibi, but God can exchange ai and bi values. At this time, ai = 2i, bi = 2i-1, while switching, a1 <a2 <... <an, b1 <b2 <... <bn, so at this time, your algorithm is wrong.

Similarly, if you do not compare ai and bi + 1, God can still exchange the values of ai and bj + 1, resulting in your algorithm being wrong, so your 2n-2 algorithm is tragic, so it proves that the lower bound is 2n-1.

For example, a list with a length of n, a1 <a2 <... <an, and a number of x are given. check whether there is an ai in this list, ai = x. Give a compare number of lower bound and prove it.

We can use the demo-tree method to find the lower bound and consider the following demo-tree:

From this tree we can conclude:

1) Each non-leaf node has two branches.

2) There are n leaf nodes in the tree (corresponding to n possible answers)

If the height of the tree is h, we know that the maximum number of leaf nodes is 2 ^ 0 + 2 ^ 1 + 2 ^ 2 +... + 2 ^ K-1, this is equal to 2 ^ K-1, of course, this is greater than or equal to n, so there is 2 ^ k-1> = n, we can get k> = lg (n + 1), which means we should compare it at least with lg (n + 1.