HEAPSORT heap Sorting Algorithm (JAVA implementation)

HEAPSORT heap Sorting Algorithm (JAVA implementation)
Heapsort (Heapsort) is one of the most classic sorting algorithms. You can search google or Baidu to find many detailed resolutions. The same good sorting algorithms include quicksort and merge sort. This algorithm is chosen for analysis mainly because it uses a very interesting algorithm technique: data Structure-heap. Heapsort is actually a sort algorithm that looks complicated but not complex. I personally think heapsort also plays an important role in machine learning. Here, I will explain all aspects of Heapsort again to consolidate this knowledge. It may be different from other articles. If there are any errors, please note. The referenced referecne in this document is marked at the end. P.s. I personally think that the so-called explanation is an easy-to-understand explanation when you read the relevant wiki or algorithm book. But the best solution is to look at the professor's explanation. We recommend the heapsort chapter in reference [1. The above is nonsense. You can skip Section 1-Overview Heapsort is a comparison-based Sorting Algorithm (fast sorting, merge, insert all; counting sort is not ), it is also a selection sort algorithm. The definition of a selection algorithm is to find the k-th order statistic (term in statistics) of a sequence ), to put it bluntly, you can find a small k-th element in the list. All of the above can be understood in general, and heapsort understands it. Let's take a look at this. Likewise, interpolation sorting is also a selection sorting algorithm. The time complexity of Heapsort is that worst-case is O (nlgn), average-case is O (nlgn), and space complexity is that worst-case is O (1 ), that is to say, heapsort can be implemented in-place; heapsort is unstable. By the way, the time complexity comparison of several sort algorithms is provided below (when-notation is more accurate than O-notation, it defines the upper and lower limits of asymptotic analysis. For more information, see google ): table 1-comparison of the running time of the four sorting algorithms Algorithm Worst-case Average-case/expected Insertion sort (interpolation sorting) Merge (n2) Merge (n2) Merge sort (Merge Sorting) sort (nlgn) sort (nlgn) Heapsort (heap sorting) O (nlgn) Quicksort (fast sorting) sort (n2) Merge (n2) (expected) * The performance of Additional Part-KNN heapsort in practice is often inferior to that of quicksort (although the worst performance of quicksort is merge (n2), runti in the case of quicksort 99% Me complexity is merge (nlgn), but the upper limit and fixed space usage of heapsort O (nlgn) are often used in embedded systems. It often plays an important role in search or machine learning. It can return only the values required for k sorting, regardless of the values of other elements. For example, in KNN (K-nearest-neighbor), you only need to return K minimum values to meet the requirements, instead of sorting globally. Of course, you can also use the divide-and-conquer idea to find the maximum/small K values. This is an aside and you will have the opportunity to make a special comparison later. The following program calls heapq in python for heapsort to obtain k minimum values, which can reflect the features described above: get_k_smallest running result: Elasped time for heapsort to return 10 smallest: 0.0350000858307 Elasped time for heapsort to return 10000 smallest: 0.0899999141693 Section 2-algorithm process understanding 2.1 binary heap full binary tree in the "heap sorting", the "heap" usually refers to "binary heap )", many informal statements say that "binary Heap" is actually a complete binary tree. This statement is correct but not accurate. However, it is very easy to understand the binary heap. The binary heap mainly meets the following two properties: #1-Shape Property: it is a complete binary tree. #2-Heap Property: mainly divided into max-heap property and min-heap property (this is the term I mentioned earlier and is very important) | -- max-heap property: for all node I except the root node, A [Parent] ≥a [I] | -- min-heap property: For all node I except the root node, the two binary tree structures in A [Parent] ≤ A [I] are full binary trees, but the right side is the binary heap that satisfies the max-heap property. In the following description, we use heapsort for the binary heap for convenience. 2.2 A preliminary conception with such A seemingly simple structure, we can produce the following preliminary conception to sort array A: 1. build A into A maximum heap (compliant with max-heap property, that is, the maximum root node); 2. how ?); 3. Create the remaining array elements into a maximum binary heap and return Step 1 until all elements are stripped. If you have already thought of the above, you have almost finished heapsort. The rest is how to use the terminology and express the algorithm in a logical and procedural manner. 2.3 There are logical and procedural expressions. Generally, heapsort uses max-heap ). Give an array A (we use the Java sequence [0... n]), We initialize it into A heap in order: Input: array A Initialization: heap Initialization * The heap root node (root) is A [0]; for the node whose index is I in this heap, We can get its parent, left child and right child, which has the following operations: Parent (I): parent (I) when A [floor (I-1)/2)] Left (I): left (I) when A [2 then I + 1] Right (I ): right (I) when A [2 then I + 2] through the above operation, we can get other nodes (parent/child) related to it in any index-I ). In heapsort, there are three very important basic operations (basic procedures): Max-Heapify (A, I): Maintain the Heap's #2-Heap Property, don't forget that in heapsort, we mean max-heap property (min-heap property is usually used to implement priority heap, which we will mention later ). Build-Max-Heap (A): As the name suggests, Build A max-heap ). Heapsort (A): Based on Build-Max-Heap (A), we implement step 2-3 in our 2.2 vision. In fact, each of these three operations is part of the subsequent operations. The following describes the three key steps in detail. The input of Max-Heapify (A, I) + Max-Heapify is the current heap A and index-I. in the actual in-place implementation, A heapsize is usually required, that is, the number of elements in the current heap. + Max-Heapify has an important assumption: the subtree with Left (I) and Right (I) as the root node is the largest heap (for example, the knowledge of fruit trees is good, and it is easy to understand here, but why is that? In the Build-Max-Heap section ). + With the preceding input and assumptions, you only need to compare A [I], A [Left (I)], and A [Right (I, there are two scenarios:-first, the maximum value (largest) is A [I], so based on the previous important assumptions, the tree with I as the root node already complies with the #2-Heap Property. -In the second case, if the maximum value (largest) is A [Left (I)] or A [Right (I)], then A [I] And A [largest] are exchanged. the result is that the subtree with largest as the root node may break the #2-Heap Property, so the Max-Heapify (A, largest) operation is performed on the tree with largest as the root node. + The operation described above has an image description called A [I] "float down", so that the tree with I as the root node complies with the #2-Heap Property, the following example shows the process of A [0] "float down" (note that the tree with A [1] and A [2] as the root node is the largest heap ). Heapify step 1 heapify step2 heapify step 3 heapify step 4 The following is attached with Psudocode in reference [1]: Copy code 1 MAX-HEAPIFY (A, I) 2 l = LEFT (I) 3 r = RIGHT (I) 4 if <= heapsize and A [l]> A [I] 5 largest = l 6 else largest = I 7 if r <= heapsize and A [r]> A [largest] 8 largest = r 9 if not largest = i10 exchange A [I] with a [largest] 11 MAX-HEAPIFY (, largest) copy the code Build-Max-Heap (A) first attach the Psudocode in reference [1] (with some modifications made, this Sample), because it is very simple: 1 BUILD-MAX-HEAP (A) 2 heapsize =. length3 for I = PARENT (. length-1) downto 04 MAX-HEAPIFY (A, I) + Build-Max-Heap first find the last node with subnodes I = PARENT (. length −1) is used as Initialization, because other nodes larger than I have no subnodes, they are all the largest heaps. + Perform a descending loop on I and perform the Max-Heapify operation on each I. Because all nodes larger than I have performed the Max-Heapify operation and the sub-nodes of I must be larger than I, it complies with the assumption of Max-Heapify (Left (I) and Right (I) are the largest heap of the subtree of the root node ). Build-Max-Heap process for our input: max heap 1 max heap 2 max heap 3 max heap 4 max heap 5 max heap 6 Heapsort () so far, we have completed the first step we have imagined in 2.2. A [0] means that the root node is the maximum value in the array. If the root node is taken out directly, the heap structure will be damaged. The heapsort algorithm uses a very clever method. + Swap root node A [0] with the last leaf node (leaf) in the heap, and then retrieve the leaf node. In this way, except for the tree with A [0] as the root, the #2-Heap Property is damaged, and the other Subtrees are still the largest Heap. You only need to perform the Max-Heapify operation on A [0. + The method for extracting the root node in this process is also very simple. You only need to extract heapsize into heapsize −1. Below is Psudocode: 1 HEAPSORT (A) in reference [1]: 2 BUILD-MAX-HEAP (A) 3 for I =. length downto 14 exchange A [0] with A [I] 5 heapsize = heapsize-16 MAX-HEAPIFY (A, 0) so far is the entire heapsort algorithm process. Note: If you want to close your eyes and write a heap, the best way is to understand the above six important operations. Section 3-runtime complexity analysis: This section analyzes the complexity of operations in the heapsort algorithm. First, we will summarize: Max-Heapify ~ O (lgn) Build-Max-Heap ~ O (n) Heapsort ~ O (nlgn) then let's analyze why it is like this. In the following analysis, all node I we refer to start from 1. Max-Heapify is not difficult to deduce. The height from any node I to the leaf node in the heap is lgn. For professional derivation, refer to using master theorem. This step is the most interesting step for building-Max-Heap to analyze the complexity of heapsort. If the heap size is n, the heap height is lgn stacking. For any node I, the height from I to the leaf node is h, then, h nodes can have at most limit n/2 h + 1 limit. The following is an intuitive proof:-first, the number of leaf nodes (leaf) in a heap with a size of n is limit n/2 bytes: -- Do not remember the last node with a child node (length-1) it is nth n/2 Second (note that this is not the java serial number, it is the nth number), it can be proved that the number of leaf nodes is n-grams n/2 rows ⌋; -If the leaf node is removed, the number of nodes in the remaining heap is n −0000n/2 bytes = 0000n/2 bytes ⌋, after this new tree is removed from the leaf node, the number of nodes is too many n/2 then/2 then;-(you need to think about it) and so on, the number of leaf nodes in the last tree is the number of nodes with h height, which must be smaller than limit (n/2)/2 h, that is, limit n/2 h + 1 hour ⌉. For any node I, the height from I to the leaf node is h, and the time required to run Max-Heapify is O (h), as shown above. Therefore, the maximum Build-Max-Heap time is (refer to reference [1]): Σ 1_lgn 1_h = 0 then n2h + 1 then O (h) = O (n Σ ⌊ lgn 1_h = 0h2h) according to the following theorem: Σ ∞ k = 0kxk = x (1−x) 2for | x | <1 we get the section where x = 12 is replaced by the sum: Σ ∞ h = 0h2h = 1/2 (1 −1/2) 2 = 2 in summary, we can obtain: O (n Σ ⌊ lgn 1_h = 0h2h) = O (n Σ ∞ h = 0h2h) = O (2n) = O (n) because Build-Max-Heap complexity is O (n), there are n-1 calls to Max-Heapify (complexity is O (lgn), and the total complexity is O (nlgn) so far, the running complexity of all functions has been analyzed. The following section describes how to use Java. Section 4-Java Implementation this Section has two contents: A simple Java Implementation (only the key sorting function) and a Priority Queue. Parameters & Constructors: copy the Code 1 protected double A []; 2 protected int heapsize; 3 4 // constructors5 public MaxHeap () {} 6 public MaxHeap (double A []) {7 buildMaxHeap (A); 8} copy the code to calculate parent/left child/right child: 1 protected int parent (int I) {return (I-1)/2 ;} 2 protected int left (int I) {return 2 * I + 1;} 3 protected int right (int I) {return 2 * I + 2;} maintain the maximum heap feature: copy the Code protected void maxHeapify (int I) {int L = left (I); int r = right (I); int largest = I; if (l <= heapsize-1 & A [l]> A [I]) largest = l; if (r <= heapsize-1 & A [r]> A [largest]) largest = r; if (largest! = I) {double temp = A [I]; // swap A [I] = A [largest]; A [largest] = temp; this. maxHeapify (largest) ;}} copy the code to construct A "Max Heap": Copy code 1 public void buildMaxHeap (double [] A) {2 this. A = A; 3 this. heapsize =. length; 4 5 for (int I = parent (heapsize-1); I> = 0; I --) 6 maxHeapify (I); 7} copy the code to use heapsort for an array: copy code 1 public void heapsort (double [] A) {2 buildMaxHeap (A); 3 4 int step = 1; 5 for (int I =. length- 1; I> 0; I --) {6 double temp = A [I]; 7 A [I] = A [0]; 8 A [0] = temp; 9 heapsize --; 10 System. out. println ("Step:" + (step ++) + Arrays. toString (A); 11 maxHeapify (0); 12} 13} copy the code main function: Copy code 1 public static void main (String [] args) {2 // a sample input3 double [] A = {3, 7, 2, 11, 3, 4, 9, 2, 18, 0}; 4 System. out. println ("Input:" + Arrays. toString (A); 5 MaxHeap maxhp = new MaxHeap (); 6 maxhp. heapsort (); 7 System. out. println ("Output:" + Arrays. toString (A); 8 9} copy the code running result: copy the code Input: [3.0, 7.0, 2.0, 11.0, 3.0, 4.0, 9.0, 2.0, 18.0, 0.0] Step: 1 [0.0, 11.0, 9.0, 7.0, 3.0, 4.0, 2.0, 2.0, 3.0, 18.0] Step: 2 [0.0, 7.0, 9.0, 3.0, 3.0, 4.0, 2.0, 2.0, 11.0, 18.0] Step: 3 [2.0, 7.0, 4.0, 3.0, 3.0, 0.0, 2.0, 9.0, 11.0, 18.0] Step: 4 [2.0, 3.0, 4.0, 2.0, 3.0, 0.0, 7.0, 9.0, 11.0, 18.0] Step: 5 [0.0, 3.0, 2.0, 2.0, 3. 0, 4.0, 7.0, 9.0, 11.0, 18.0] Step: 6 [0.0, 3.0, 2.0, 2.0, 3.0, 4.0, 7.0, 9.0, 11.0, 18.0] Step: 7 [0.0, 2.0, 2.0, 3.0, 3.0, 4.0, 7.0, 9.0, 11.0, 18.0] Step: 8 [2.0, 0.0, 2.0, 3.0, 3.0, 4.0, 7.0, 9.0, 11.0, 18.0] Step: 9 [0.0, 2.0, 2.0, 3.0, 3.0, 4.0, 7.0, 9.0, 11.0, 18.0] Step: 10 [0.0, 2.0, 2.0, 3.0, 3.0, 4.0, 7.0, 9.0, 11.0, 18.0] Output: [0.0, 2.0, 2.0, 3.0, 3.0, 4.0, 7.0, 9.0, 11.0, 18.0] copy the code heapsort in Practice is often defeated by a fast implementation, but heap has another important application, namely, Priority Queue. This article only mentions extended content. To put it simply, a priority queue is a group of elements with keys and uses keys to construct the heap structure. In general, priority queue uses min-heap, for example, processing objects in certain applications in chronological order. For convenience, I use Inheritance to implement a priority queue: ackage heapsort; 2 3 import java. util. arrays; 4 5 public class PriorityQueue extends MaxHeap {6 7 public PriorityQueue () {super ();} 8 public PriorityQueue (double [] A) {super ();} 9 10 public double maximum () {11 return A [0]; 12} 13 14 public double extractMax () {15 if (heapsize <1) 16 System. err. println ("no element in the heap"); 17 double max = A [0]; 18 A [0] = A [heapsize-1]; 19 heapsize --; 20 this. maxHeapify (0); 21 return max; 22} 23 24 public void increaseKey (int I, double key) {25 if (key <A [I]) 26 System. err. println ("new key shocould be greater than old one"); 27 28 A [I] = key; 29 while (I> 0 & A [parent (I)] <A [I]) {30 double temp = A [I]; 31 A [I] = A [parent (I)]; 32 A [parent (I)] = temp; 33 I = parent (I); 34} 35} 36 37 public void insert (double key) {38 heapsize ++; 39 A [heapsize-1] = Double. MIN_VALUE; 40 increaseKey (heapsize-1, key); 41} 42 43 public static void main (String [] args) {44 // a sample input45 double [] A = {3, 7, 2, 11, 3, 4, 9, 2, 18, 0}; 46 System. out. println ("Input:" + Arrays. toString (A); 47 PriorityQueue pq = new PriorityQueue (); 48 pq. buildMaxHeap (A); 49 System. out. println ("Output:" + Arrays. toString (A); 50 pq. increaseKey (2,100); 51 System. out. println ("Output:" + Arrays. toString (A); 52 System. out. println ("maximum extracted:" + pq. extractMax (); 53 pq. insert (33); 54 System. out. println ("Output:" + Arrays. toString (A); 55 56} 57} copy the code running result: Input: [3.0, 7.0, 2.0, 11.0, 3.0, 4.0, 9.0, 2.0, 18.0, 0.0] Output: [18.0, 11.0, 9.0, 7.0, 3.0, 4.0, 2.0, 2.0, 3.0, 0.0, 100.0] Output: [11.0, 18.0, 7.0, 3.0, 4.0, 2.0, 2.0, 3.0, 0.0] maximum extracted: 100.0 Output: [33.0, 18.0, 4.0, 7.0, 11.0, 0.0, 2.0, 2.0]