How to filter out the top 100,000 minimum values in a large number of data (e.g. 100 million)? The second

Prepare the data that can be read into memory at once, take the former K small, in a variety of ways. Imagine: Insert sort, binary find insert sort, chain sort, heap sort.

Sequential Lookup Direct insertion:

Ideas:

1, the entire data, in turn, to take the value. Sent to a queue.

2, to judge the data, in the queue to do a sequential search. Insert if location is found.

3. When the queue is full and the data to be judged is greater than the maximum data in the queue, it is discarded directly.

In addition, in the code, the result of C + + is compared with the value of this program to determine the correct results.

The code is as follows:

voidSort1 (intData[],intMintNintout[]) {   for(intCount=0, i=0; i< m; i++)    if(count== n && data[i]>= out[n-1])      Continue;//invalid data dropped directly    Else       for(intj=0; ; J + +)//Sequential Lookup        if(j== count | | data[i]<= Out[j])//Find insertion point        {           for(intk= count; K> J; k--)//empty the insertion pointout[k]= out[k-1]; OUT[J]=Data[i]; if(count<N) Count++;//Increase Queue Length           Break; }}voidShowTime (String caption, Double &TMS) {ShowMessage (String (Caption+"   "+ (GetTickCount ()-TMS)/ +)+"seconds"); TMS=GetTickCount ();}voidControlintN) {  intm= N Pow (Ten,3); Doubletms=GetTickCount (); int*data=New int[M], *datasort=New int[M]; int*out=New int[n+1];  for(intI=0; i< m; i++)//random numbers.datasort[i]= data[i]=random (m); ShowTime ("When making random numbers", TMS); Sort (Datasort, Datasort+m); ShowTime ("Standard sorting time", TMS);  Sort1 (Data, M, N, out); ShowTime ("When sorting", TMS);  for(intlim= N, i=0; I<= Lim; i++)    if(i== Lim | | datasort[i]!=Out[i]) ShowMessage (i= = Lim?"Correct value":"error in fetching value"), i=Lim; Delete[]datasort; Delete[]data; Delete[]out;}

When the function control (int n), the n value is 100,000, the time is as follows:

Manufacturing random data: 8.346 seconds;

C + + calibration sequencing: 75.864 seconds;

This procedure value: 1121.289;

Time is not absolutely meaningful. But, later, write more methods. The time between them can be compared.

void Sort2 (int data[], int m, int n, int out[]) {for  (int count= 0, i= 0; i< m; i++)    if (count== n && Dat a[i]>= out[n-1])      continue;    else    {      int mid, low= 0, high= count-1;      for (; mid= (high+ Low) >> 1, low<= high;)                 Binary Find        if (data[i]>= Out[mid]) low= mid+ 1; else high= mid-1;      for (int k= count, k> Low, k--)                            //Empty insertion point        out[k]= out[k-1];      out[low]= Data[i];      if (count< N)        count++;  }}

Binary Find, Time:

Standard: 53.8 seconds;

Value: 278.508 seconds.

From a standard sort of view, it seems, this time, the program runs faster.

How to filter out the top 100,000 minimum values in a large number of data (e.g. 100 million)? The second