How can we sort 10 GB files (one number per row) in MB of memory? How can I find 10 GB files? How does one calculate the number of times each keyword appears in a 10g File? How does one sort 10G files by MB in memory (one number per line? How can I find 10 GB files? How to calculate the number of times each keyword appears in a 10 GB file
Reply content:
How can we sort 10 GB files (one number per row) in MB of memory? How can I find 10 GB files? How to calculate the number of times each keyword appears in a 10 GB file
Use time for space
The specific implementation is to load files in batches and then calculate
Java? using nio and mapreduce
Not understandPhp
But I am familiar with reading this question.
Let's talk about the idea.
1. implementation of sorting
This is a typical topic of standalone external sorting. The specific method isSort parts first
ThenMerge multiple channels
Output file.
2. search
If the file cannot be processed, you can only traverse and search for the file.
If the file can be processed, the file has been sorted and can be processed.Binary search
.
3. Statistics
If the file cannot be processed, there is still no good way to traverse it.
If the order has been sorted, you can directly perform binary search. Search for the number at the specified position.
Let's take a look at the book "programming Pearl", which seems to be a problem.