"In-depth study of Javanio 4" Memory mapped file I/O, large file read and write operations, Java NiO mappedbytebuffer, efficient file/memory mapping

Memory-mapped files allow you to create and modify files that are too large to fit into memory. With a memory-mapped file, you can assume that the file has all been read into memory, and then it is accessed as a very large array. This solution greatly simplifies the code to modify the file.
Filechannel.map (Filechannel.mapmode mode, long position, long size) maps the file area of this channel directly into memory. Note that you have to indicate where it starts mapping from where the file is, how big the map is, and that is, it can also map a small fragment of a large file.

Mappedbytebuffer is a subclass of Bytebuffer, so it has all the methods of Bytebuffer, but new force () forces the contents of the buffer to be flushed to the storage device, load () loads the data from the storage device into memory, IsLoaded () The data in memory is synchronized with the storage settings. In addition to the put () and get () methods, you can easily read and write basic type data by using methods such as Ascharbuffer () to get a buffered view of the corresponding basic type of data.

Memory-mapped file I/O is a way to read and write file data, which can be much faster than regular stream-based or channel-based I/O.

Memory-mapped file I/O is done by making the data in a file magically appear as the contents of a memory array. This may sound like an entire file being read into memory, but it's not in the first place. In general, only the portions of the file that are actually read or written are fed (or mapped ) into memory.

Memory mapping is not really magical or unusual. Modern operating systems typically map portions of a file to a portion of memory as needed to implement a file system. The Java memory mapping mechanism provides access to this mechanism only when it is possible to use this mechanism in the underlying operating system.

Although it is fairly straightforward to create a memory-mapped file, writing to it can be dangerous. Simply manipulating a single element of an array can directly modify the files on the disk. There is no separation between modifying the data and saving the data to disk.

(1) The best way to understand memory mapping is to use examples. In the example below, we are going to FileChannel map one (all or part of it) into memory. For this we will use the FileChannel.map() method. The following code lines map the first 1024 bytes of the file into memory:

public class usemappedfile{  static private final int start = 0;  static private final int size = 1024x768;  static public void Main (String args[]) throws Exception {    Randomaccessfile RAF = new Randomaccessfile ("Usemappedfil E.txt "," RW ");    FileChannel FC = Raf.getchannel ();    Mappedbytebuffer MBB = Fc.map (FileChannel.MapMode.READ_WRITE,      start, size);    Mbb.put (0, (byte));    Mbb.put (1023, (byte) 122);    for (int i = start; i < size; i++) {           System.out.print ((char) mbb.get (i));      }      Raf.close ();  }}

Operation Result:

(2) The program created a 128Mb file, if a one-time read to memory may lead to memory overflow, but the access here seems to be just a moment, this is because the real transfer into the memory is only a small part of it, the rest is placed on the Exchange file. This makes it easy for you to modify very large files (up to 2 GB). Note that Java is called the "file mapping mechanism" of the operating system to improve performance.

public class Largemappedfiles {      static int length = 0x8000000;//-Mb public        static void Main (string[] args) th Rows Exception {          //In order to open a file in a readable and writable manner, the randomaccessfile is used to create the file.          FileChannel fc = new Randomaccessfile ("Test.dat", "RW"). Getchannel ();          Note that the readable writable file channel is based on the file stream itself can be read and written on the basis of          mappedbytebuffer out = Fc.map (FileChannel.MapMode.READ_WRITE, 0, length);          Write 128M content for          (int i = 0; i < length; i++) {              out.put ((Byte) ' x ');          }          System.out.println ("finished writing");          Read the middle of the file 6 bytes of content for          (int i = LENGTH/2; I < LENGTH/2 + 6; i++) {              System.out.print (char) out.get (i));          }          Fc.close ();      }  }  

Operation Result:

Finished writing
Xxxxxx

(3) Java processing large files, generally with bufferedreader,bufferedinputstream this kind of buffered IO class, but if the file is very large, the faster way is to use Mappedbytebuffer.

Three different ways:
FileChannel provides a map method for mapping files to memory image files: Mappedbytebuffer map (int mode,long position,long size); You can map the size of a file from position to a memory image file, mode indicates how the memory image file can be accessed: read_only,read_write,private.

A. Read_Only, (read-only): Attempting to modify the resulting buffer will cause the readonlybufferexception to be thrown. (mapmode.read_only)

B. Read_write (read/write): Changes to the resulting buffer will eventually propagate to the file; The change is not necessarily visible to other programs that map to the same file. (Mapmode.read_write)

C. Private (private): changes to the resulting buffer are not propagated to the file, and the change is not visible to other programs that map to the same file; instead, a private copy of the modified portion of the buffer is created. (mapmode.private)

Three methods:

A. Fore (); buffer is Read_write mode, this method forces the file to be written to the buffer content modification
B. Load () Loads the contents of the buffer into memory and returns a reference to the buffer
C. isLoaded () If the contents of the buffer are in physical memory, returns true, otherwise false

Three features:

After calling the channel's map () method, you can map part or all of the file to memory, the mapped memory buffer is a direct buffer and inherits from Bytebuffer, but it has more advantages over Bytebuffer:

A. Read fast
B. Write fast
C. Write Anywhere, anytime

Package study;  Import Java.io.FileInputStream;  Import Java.io.FileOutputStream;  Import Java.nio.ByteBuffer;  Import Java.nio.MappedByteBuffer;    Import Java.nio.channels.FileChannel; public class Mapmemerybuffer {public static void main (string[] args) throws Exception {Bytebuffer Bytebu          F = bytebuffer.allocate (1024 * 14 * 1024);          byte[] bbb = new BYTE[14 * 1024 * 1024];          FileInputStream fis = new FileInputStream ("E://data/other/ultraedit_17.00.0.1035_sc.exe");          FileOutputStream fos = new FileOutputStream ("E://data/other/outfile.txt");          FileChannel FC = Fis.getchannel (); Long Timestar = System.currenttimemillis ();//Get Current time Fc.read (BYTEBUF);//1 read//mappedbytebuffer MBB =          Fc.map (FileChannel.MapMode.READ_ONLY, 0, Fc.size ());          System.out.println (Fc.size ()/1024); Long timeend = System.currenttimemillis ();//Gets the current time System.out.println ("Read Times:" + (Timeend-timestar) + "M     S ");     Timestar = System.currenttimemillis ();          Fos.write (BBB);//2. Write//mbb.flip ();          Timeend = System.currenttimemillis ();          System.out.println ("Write Time:" + (Timeend-timestar) + "MS");          Fos.flush ();          Fc.close ();      Fis.close (); }} Run Result: 14235 Read time:24ms Write time:21ms We commented out the annotations 1 and 2, replaced them with the annotated statement below, and then looked at the effect of the operation.   14235 Read time:2ms Write time:0ms

It can be seen that the speed has been greatly improved. Mappedbytebuffer is fast, but there are some problems, such as memory footprint and file shutdown. Files opened by Mappedbytebuffer are closed only when garbage is collected, and this point is indeterminate. That's what it says in Javadoc: a mapped byte buffer and the file mapping that it represents remain valid until the buffer itself is Garba Ge-collected.

"In-depth study of Javanio 4" Memory mapped file I/O, large file read and write operations, Java NiO mappedbytebuffer, efficient file/memory mapping