Windows core programming-memory ing File

Like the virtual memory, the memory ing file can be used to reserve an address space area and submit the physical memory to the area. The difference between them is that physical storage comes from a file already on the disk, rather than a system page file. Once the file is mapped, you can access it, just as the entire file has loaded memory.

The memory ing file can be used for three different purposes:

• The system uses memory ing files to load and execute. e x E and d l files. This greatly saves the page file space and the time required for the application to start and run.

• You can use a memory ing file to access data files on the disk. This eliminates the need to perform I/O operations on files and cache file content.

• Memory ing files can be used to allow multiple processes running on the same computer to share data with each other. Wi n d o w s does provide other methods for data communication between processes, but these methods are implemented using memory ing files, this makes the memory ing file the most effective way for multiple processes on a single computer to communicate with each other.

Executable and DLL files mapped by memory

When the thread calls C r e a t e p r o c e s, the system performs the following steps:

1) The system finds the. e x e file set when calling C r e a t e p r o c e s. If the. e x e file cannot be found, the process cannot be created. C r e a t e p r o c e s will return fa l e.

2) The system creates a new process kernel object.

3) The system creates a private address space for the new process.

4) The system reserves an address space area that is large enough to store the. e x e file. The location required for this region is set in the. e x e file itself. Follow the default settings ,. the base address of the e x e file is 0x0 4 0 0 0 0 0 0 (this address may be different from the address of the 6 4-bit application running on 6 4-bit Windows 2000), however, you can. this address is reloaded when the e x e file is used to use the/B A S E Option of the linked program when linking the application.

5) The system notices that the physical storage that supports reserved areas is in the. e x e file on the disk, rather than in the system page file.

When. after the e x e file is mapped to the address space of the process, the system will access it. A part of the E x e file, which contains. the d l file of the function to be called by the code in the E x e file. Then, the system calls the l o a d l I B R A R Y function for each d l file. If any d l requires more d l, then the system will call the l o a d l I B R A R Y function to load these d L. Every time you call l o a d l I B R A R Y to load a d l, the system performs the following steps, which are similar to steps 4th and 5th above:

1) The system reserves an address space area that is large enough to store the d l file. The location required for this region is set in the d l file itself. Follow the default settings, the base address of the d l file created by Visual C ++ of m I c r o s o f t is 0x1 0 0 0 0 0 0 0 0 0 0 (this address may be different from 6 4-bit Windows 2000 running 6-bit d l address) however, you can reload the address when creating the d l file by using the/B A S E Option of the linked program. All the standard system d l provided by wi n d o w s has different base addresses, so that they do not overlap if they are loaded into a single address space.

2) If the system cannot retain a region on the preferred base address of the d l, it may be because the region has been replaced by another d l or. e x e is occupied, or it may be because the area is not large enough. At this time, the system will try to find another address space area to retain the d L.

3) The system will notice that the physical storage that supports the reserved area is located in the d l file on the disk, rather than in the system's page file.

If the system cannot be mapped for a reason. e x e and all necessary d l files, the system displays a message box to the user and releases the address space and process object of the process.
After all. e x E and d l files are mapped to the address space of the process, the system can start executing the startup code of the. e x e file. When the. e x e file is mapped, the system will be responsible for all paging, buffering, and cache processing.
Share static data between multiple instances of executable files or DLL(By defining shared sections)

 

 

 

Global data and static data cannot be shared by multiple images of the same. e x e or d l file. This is a safe default setting. However, in some cases, it is very useful and convenient to share an instance of a variable with multiple images of A. e x e file. For example, wi n d o w s does not provide any easy way to determine whether the user is running multiple instances of the application. However, if you want all instances to share a single global variable, this global variable can reflect the number of running instances.
Memory ing Data File

The operating system allows the memory to map a data file to the address space of the process. Therefore, it is very convenient to operate on a large amount of data.

To understand how to use the memory ing file function in this way, let's take a look at how to use four methods to implement a program so as to reverse the order of all the bytes in the file.

Method 1: one file and one cache

The first method is also the simplest method in theory. It needs to allocate enough memory blocks to store the entire file. The file is opened, its content is read into the memory block, and the file is closed. After the file content enters the memory, we can reverse the order of all bytes. The method is to reset the first byte to the last byte, and the second byte to the last byte, and so on. This reset operation will continue until the intermediate position of the file. After all the bytes have been reset, you can re-open the file and rewrite its content with the content of the memory block.

This method is very easy to implement, but it has two disadvantages. First, you must allocate a memory block of the same size as the file. If the file is small, there is no problem. But what should I do if the file is very large, for example, 2g B? A 3-2-bit system does not allow applications to submit such large physical memory blocks. Therefore, different methods are required for large files.

Second, if the process is interrupted during the running process, that is, when the inverted byte is re-written to the file, the process is interrupted, and the file content is damaged. The simplest way to prevent this is to create a copy of the original file before the reverse order of its content. If the entire process runs successfully, you can delete the copy of the file. This method requires more disk space.

Method 2: two files, one cache

In method 2, you open an existing file and create a new file with a length of 0 on the disk. Then allocate a small internal cache, such as 8 KB. You can find a location 8 KB away from the end of the original file, read the last 8 KB into the cache, and write the cached content into the new file in reverse order. This search, read, reverse, and write operations must be performed repeatedly until the beginning of the original file is reached. If the file length is not a multiple of 8 KB, some special processing is required. After the original file is fully processed, close the original file and the new file and delete the original file.

This method is more complicated than the first method. The memory usage efficiency is much higher, because it only needs to allocate an 8 KB cache block, but it has two major problems. First, the processing speed is slower than that of the first method, because in each loop operation, the original file must be searched before the read operation is executed. Second, this method may require a large amount of hard disk space. If the original file is 400 mb, the new file will increase to 400 mb as the process continues to run. Before the original file is deleted, the two files occupy a total disk space of 800 mb. This is 400 mb larger than the required space. This disadvantage leads to the next method.

Method 3: one file and two caches

If this method is used, we assume that two independent 8 KB caches are allocated during program initialization. The program reads the first 8 KB of the file into one cache, and then the second 8 KB of the file into another cache. Then, the process reversely sorts the two cached content, writes the first cached content back to the end of the file, and writes the second cached content back to the beginning of the same file. Each iteration is performed continuously (in the unit of 8 KB, the file block is moved from the beginning and end of the file ). If the file length is not a multiple of 16 KB and two 8 KB file blocks overlap, special processing is required. This kind of special processing is more complicated than the special processing in the previous method, but it is not difficult for experienced programmers.

Compared with the previous two methods, this method has its advantages in saving hard disk space. Because all content is read from the same file and written to the same file, no extra disk space is required. As for memory usage, this method is also good, it only needs 16 KB of memory. Of course, this method may be the most difficult to implement. Like the first method, if the process is interrupted, this method will cause the data file to be damaged.

Next let's take a look at how to use the memory ing file to complete this process.

Method 4: a file with no cache

When you use a memory ing file to reverse the file content, open the file and tell the system to reverse the area of the virtual address space. You tell the system to map the first byte of the file to the first byte of the reserved area. Then you can access the region of the virtual memory, just as it contains the file. In fact, if there is a single 0-byte at the end of the file, you only need to call the C Runtime function _ s t r e v to perform reverse operations on the data in the file.

The biggest advantage of this method is that the system can manage all the File Cache operations for you. You do not need to allocate any memory, load file data to the memory, or re-write data to the file, or release any memory blocks. However, the memory ing file may still cause data corruption due to process interruptions such as power failure.

Use memory ing files

To use a memory ing file, follow these steps:

1) Create or open a file kernel object to identify the file on the disk that you want to use as the memory ing file.

2) create a file ing kernel object to tell the system the file size and how you plan to access the file.

3) Let the system map all or part of the file ing object to your process address space.

When using the memory ing file, you must perform the following steps to clear it:

1) Tell the system to cancel the image mapped to the kernel object from the address space of your process.

2) disable the file ing kernel object.

3) close the file kernel object.

Step 1: Create or open a file Kernel Object

To create or open a file kernel object, always call the C r e a t e f I l e function:

 

HANDLE CreateFile(   PCSTR pszFileName,   DWORD dwDesiredAccess,   DWORD dwShareMode,   PSECURITY_ATTRIBUTES psa,   DWORD dwCreationDisposition,   DWORD dwFlagsAndAttributes,   HANDLE hTemplateFile);

Step 2: create a file ing Kernel Object

Call the C r e a t e f I l e function to tell the operating system the location of the physical memory of the file image. The path name you pass indicates the exact location of the physical storage that supports file images on a disk (or network or disc. In this case, you must tell the system how much physical storage is required for the file ing object. To perform this operation, you can call the C r e a t e I l e m a p I n g function:

 

HANDLE CreateFileMapping(   HANDLE hFile,   PSECURITY_ATTRIBUTES psa,   DWORD fdwProtect,   DWORD dwMaximumSizeHigh,   DWORD dwMaximumSizeLow,   PCTSTR pszName);

Step 3: map the file data to the address space of the process

After a file ing object is created, the system must retain an address space area for the file data and submit the file data as a physical storage mapped to the region. You can call the m a p Vi e w o f I l e function to perform this operation:

 

PVOID MapViewOfFile(   HANDLE hFileMappingObject,   DWORD dwDesiredAccess,   DWORD dwFileOffsetHigh,   DWORD dwFileOffsetLow,   SIZE_T dwNumberOfBytesToMap);

Step 4: remove the file data image from the address space of the process

When you no longer need to retain the file data mapped to your process address space, you can release it by calling the following function:

 

BOOL UnmapViewOfFile(PVOID pvBaseAddress);

To increase the speed, the system caches the data page of the file and does not immediately update the disk image of the file when operating on the ing view of the file. If you need to ensure that your updates are written to the disk, you can force the system to re-write part or all of the modified data to the disk image, the method is to call the f l u s h vi e w o f I l e function:

 

BOOL FlushViewOfFile(   PVOID pvAddress,   SIZE_T dwNumberOfBytesToFlush);

Step 5 and Step 6: Disable the file ing object and file object

Needless to say, you always need to close the kernel object you opened. If you forget to close it, the resource leakage will occur when your process continues to run. Of course, when your process stops running, the system will automatically shut down any objects that your process has opened but you forgot to close. However, if your process is not terminated for the moment, you will accumulate many resource handles. Therefore, you should always write clear and "correct" code to close any opened object. To disable the file ing object and file object, you only need to call the c l o s e h a n d l e function twice, and each handle calls the function once:

Let's take a closer look at this process. The following pseudo code shows an example of a memory ing file:

 

HANDLE hFile = CreateFile(...);HANDLE hFileMapping = CreateFileMapping(hFile, ...);PVOID pvFile = MapViewOfFile(hFileMapping, ...);// Use the memory-mapped file.UnmapViewOfFile(pvFile);CloseHandle(hFileMapping);CloseHandle(hFile);

The code above shows the "expected" method used to operate the memory ing file. However, it does not show the increasing number of file objects and file ing objects when you call m a p Vi e w o f I l e. This side effect is very large, because it means we can rewrite the above Code segment as shown below:

 

HANDLE hFile = CreateFile(...);HANDLE hFileMapping = CreateFileMapping(hFile, ...);CloseHandle(hFile);PVOID pvFile = MapViewOfFile(hFileMapping, ...);CloseHandle(hFileMapping);// Use the memory-mapped file.UnmapViewOfFile(pvFile);

When operating on memory ing files, you usually need to open the file, create a file ing object, and then map the data view of the file to the address space of the process using the file ing object. As the system increments the internal usage count of file objects and file ing objects, you can close these objects when your code starts to run to eliminate the possibility of resource leakage.

If you use the same file to create more file ing objects or map multiple views of the same file ing object, therefore, you cannot call the c l o s e h a n d l e function earlier. You may need to use their handles later, so that more calls can be made to C r e a t e I l e m a p I n g and m a p Vi e w o f I l e functions respectively.

Use memory ing files to handle large files
Use a memory ing file to share data between processes

Wi n d o w s always offers a variety of mechanisms that allow applications to quickly and conveniently share data and information. These mechanisms include r p c, c o m, o l e, d e, and window messages (especially w m _ c o p y d ata), clipboard, mailbox, pipeline, socket, etc. In wi n d o w s, the lowest layer mechanism for sharing data on a single computer is the memory ing file. Yes. If all processes that communicate with each other are on the same computer, all the mechanisms mentioned above use memory ing files for their tedious work. If high performance and low overhead are required, the memory ing file is the best mechanism available for raising hands.

The data sharing method is implemented by ing two or more processes to the view of the same file ing object, which means that they will share the same page of the physical storage. Therefore, when a process writes data to a view of a shared file ing object, other processes can immediately see the data changes in their views. NOTE: If multiple processes share a single file ing object, all processes must use the same name to indicate the file ing object.

Let's look at an example to start an application. When an application starts, the system calls the C r e a t e f I l e function to open the. e x e file on the disk. Then, the system calls the C r e a t e f I l e m a p I n g function to create a file ing object. Finally, the system calls the m a p Vi e w o f I l e x function on behalf of the newly created process (it carries the s e c _ I m a G E sign ), in this way ,. the E x e file can be mapped to the address space of the process. Here we call m a p Vi e w o f I l e x, instead of m a p Vi e w o f I l e, the file image will be mapped to. the base address in the E x e file image. The system creates the main thread of the Process, puts the address of the first byte of the executable code in the ing view into the instruction pointer of the thread, and then starts running the code in c p u.

If the user runs the second instance of the same application, the system considers it as required. the E x e file already has a file ing object, so no new file object or file ing object will be created. On the contrary, the system maps a view of the file for the second time, which is mapped in the address space environment of the newly created process. The system maps the same file to two address spaces at the same time. Obviously, this is more effective for memory usage, because the two processes will share the same page that contains the physical memory of the part of the Code being executed.

Like all kernel objects, you can use three methods to share objects with multiple processes. These three methods are handle inheritance, handle naming, and handle replication.
Correlation between memory ing files and data views

Appendix:

The system allows you to map multiple views of the same data of a file. For example, you can map the first 10 KB of a file to a view, and then map the first 4 kb of the same file to another view. As long as you are ing the same file ing object, the system will ensure the relevance of the mapped View data. For example, if your application changes the file content in a view, all other views are updated to reflect this change. This is because, although the page is mapped to the virtual address space of the process multiple times, the system only places data on a single r a m page. If multiple processes map views of a single data file, the data is still related, because in the data file, each r a m page has only one instance, which maps the r a m page to the address space of multiple processes.

Note that wi n d o w s allows you to create several file ing objects supported by a single data file. Wi n d o w s cannot ensure that the views of these different file ing objects are correlated. It can only ensure that multiple views of a single file ing object are correlated.

However, when operating on files, there is no reason for another application to call the C r e a t e f I l e function to open the same file mapped by another process. This new process can use the r e a D f I l e and wR I t e f I l e functions to read the data of the file and write the data into the file. Of course, every time a process calls these functions, it must read file data from the memory buffer or write file data into the memory buffer. The memory buffer must be a buffer created by the process itself, rather than the memory buffer used by the ing file. When two applications open the same file, the problem may occur: A process can call the r e a D f I l e function to read a part of the file, modify the data, and then use the WR I t e f I l e function to re-write the data to the file, the file ing object of the second process does not know the operations performed by the first process. For this reason, when you call the C r e a t e f I l e function for a file mapped to the memory, it is best to set the value of the d w s h a r e m o d e parameter to 0. In this way, you can tell the system that you want to access this file separately, and other processes cannot open it.

Read-Only files do not have relevance issues, so they can be used as good memory ing files. Memory ing files should never be used to write files on the shared network, because the system cannot guarantee the relevance of the data view. If a computer updates the file content, other computers with raw data in the memory will not know that the information has been modified.