Introduction to the MIT algorithm--the first. Analysis of algorithm

The topic of introduction to MIT algorithm under this column (algorithms) is an individual's learning experience and notes on the introduction to the MIT algorithm of NetEase Open course. All the content comes from the lectures of Charles E. Leiserson and Erik Demaine teachers in MIT Open Course Introduction to algorithms. (http://v.163.com/special/opencourse/algorithms.html)

Section I-------Course introduction and algorithm analysis of algorithm

Algorithmic Analysis: A theoretical study on the efficiency and resource utilization of computer programs.

The content of the first lesson is relatively simple, but Daniel is Daniel, can be the abstract things to say the very image, let people have a photographic memory at the same time to think behind the principle. To summarize, there are a few points of knowledge in the main.

1. The main content of the algorithm analysis is efficiency, is more important than efficiency? Of course there are, for example, correctness, maintainability, extensibility, robustness, security, and so on, and you can imagine a lot of the important things that have been discussed in software engineering before. Why, then, is the efficiency of the algorithm analyzed? In class, the teacher said a very interesting analogy, do you think money is important or water and rice important? Of course, water and rice, money is not guaranteed to survive, but money can be changed to water and rice. The "Efficiency" in algorithm analysis is equivalent to "money", you can use "efficiency" in exchange for other things, such as security, stability and so on. It's just a barter, but we can't live without it.

2. Factors that measure efficiency. Since efficiency is so important, what factors do we use to measure the efficiency of a program? Run time. For the same set of inputs, your program runs shorter than others, which means your program is more efficient on this set of datasets than others.

There are three factors that need to be considered for running time:

A, the data input situation. For example, for an insert sort algorithm, an already sequenced sequence is easier to sort;

b, the size of the data. It is clear that short sequences are easier to sort than long sequences;

C. Find the upper bound of the run time. In general, we need to find out how long the run time is for the worst input data in this program. After all, everyone wants a guarantee.

3. Several methods of analyzing run time t (N).

Worst-case: (usually)--use T (n) to indicate the maximum run time of the algorithm when the input size is n. Its role is that you can use it to make promises to others that my algorithm will run no more than T (N) in the worst case scenario.

Average-case: (sometimes)--use T (n) to represent an expectation of the algorithm's run time in all sequences of input size n. Of course, its premise is to assume the statistical probability distribution of the input, that is, for a size of n input data, it all the arrangement of the probability of the occurrence is equal.

Best-case: (bogus)--and if you want to cheat and run on a very inefficient algorithm with a good set of data, we call it the best case of the algorithm running time, which is not convincing enough.

4.Big idea--Asymptotic analysis.

There is a difference between the relative time and the absolute time that we usually call running time. For example, in a giant machine and running the same program on a microcomputer, the time displayed is different. This is what we need to introduce a more macroscopic concept: Asymptotic analysis-for an algorithm's running time, ignoring those that depend on the machine, ignoring all the low-order items, only the highest order, and focusing on the growth of the runtime, not just the run time. Introduce a mnemonic symbol θ (n), for example: If an algorithm runs for: 3n^3 + 2n^2 + 4n + 1, then ignore the constant 1 of dependent machines, and all low order items 2n^2, 4n, then the time complexity of this algorithm is θ (n^3).

Here, the teacher also carried out the very image of the explanation. If the asymptotic time complexity of algorithm A is θ (n^3), and the algorithm B is θ (n^2), then there must be a large enough n, so that when the data scale is greater than N, the running time of the algorithm B is less than a, no matter how much the advantages of the algorithm a start, regardless of the asymptotic complexity of the algorithm B coefficients and constants is useless. With such a mnemonic, the analysis of the time complexity can be independent of the machine, independent of the specific constants, the analysis algorithm will be very advantageous to us.

5. Two examples--insert sort, merge sort

The idea of inserting a sort is, for each a[i], consider its appropriate insertion position k in a[1...i-1], and then move the a[k...i-1] back one position in turn, and insert a[i] into the a[k] position.

This is the pseudo-code that inserts the sort, using indentation to represent the hierarchy of the algorithm. Make an asymptotic analysis of the insertion sort, as shown in. The following series of analysis procedures show that the asymptotic complexity of the insertion sort is at the n^2 level.

The so-called merge sort, for example, the two tables of the merge sort recursive processing are already in order, for {2, 7, 13, 20} and {1, 9, 11, 12}.

How do we merge the two tables? The two tables are already ordered, and we want to find the smallest element left in the current table, the smallest element must be the head of Table 1 or table 2, and then take it out and put it in the total table, when we remove the 1 element. Then there are 3 elements left in table 2, then compare table 1 and table 2 to find the smallest element, 2, so that we can sequentially find the two ordered table to be merged into an ordered table. As shown in.

is the pseudo-code of merge sort and algorithm asymptotic analysis.

Below we use recursive tree method to analyze the running time of the algorithm, first write the recursive expression of the run time: if we consider the case of n>1, T (n) = 2T (N/2) + Cn, where C is a constant. The form of a recursive tree is drawn as shown in the following:

So, let's take a look at the third picture above, the time of the first line and the time for Cn, the second line and The Times for C (N/2) +c (N/2) = Cn, ..., and for the time of the LGN row and for θ (n). Among them, LGN is the height of the tree, simple analysis can be drawn.

So altogether LGN line, each line time and for Cn, simple calculation can know the total time of the recursive tree and (that is, the algorithm's asymptotic time and) is (Cn) LGN +θ (n), ignoring the low order θ (n), that is, the algorithm's asymptotic time complexity is θ (NLGN).

In this case, the contents of the first section are almost over. At the end of the day, the teacher gave a conclusion: the merge sort can be more efficient when the input size n increases in the time of the asymptotic more than the insertion sort, in the worst case. In fact, when n>30 and above, the efficiency of merge sort is higher than the efficiency of inserting sort. Go test it out for yourself!

More learning materials about introduction to algorithms will continue to be updated, so stay tuned for this blog and Sina Weibo Sheridan.

Introduction to the MIT algorithm--the first. Analysis of algorithm