Windows CE process, thread, and memory management (3)

Iii. Memory Management

Like other Windows operating systems, Windows CE. Net also supports 32-bit virtual memory, on-demand memory allocation, and memory ing files. However, it is significantly different from other Windows operating systems. After all, Windows CE is an embedded real-time operating system. In terms of memory management, it must save more physical memory and virtual address space than other Windows operating systems. In terms of Memory Management APIsProgram, Windows CE and other Windows operating system function declarations are basically the same, which allows a programmer developed in other windows to directly use the familiar API functions, however, developers should be familiar with the principle of memory management under CE.

1. Rom and Ram
The earliest Windows CE-based civilian product uses Rom + RAM as the storage devices. Rom stores ce kernel files and applications, while Ram is used in the kernel and when all applications are running, when power off, Ram must provide power to save system configuration information and user generated files. To adapt to such storage hardware, Ce uses the ROM file system and ram file system. The modules stored in the ROM can be compressed or not compressed, depending on the OEM. The OEM can set whether to compress the module during kernel customization. For compression, the module decompress the package before running and store all the files in Ram. If it is not compressed, execute it locally (xip, executed in place ). The local execution is the same as that of other Windows operating systems to execute applications and DLL, that is, the memory ing file technology is applied. By the way. Application or DLL Code Segments are not loaded into the physical memory. The kernel only allocates virtual address space to segments. When code is executed, the kernel searches for code and executes the code in the files actually stored on the hard disk. Using this technology can save both the available memory and the loading time. Note that the operating system will first read the code data in the buffer prepared for the hard disk. If no code data is available, it will command the hard disk to read the application file data to the buffer. Therefore, it is advantageous to set a greater value for the buffer zone. The local execution of Windows CE uses this technology to load the applications and DLL in the Rom. Therefore, Windows CE dll can be divided into xip DLL and non-xip DLL. The disadvantage of this loading method is that the execution is relatively slow. If you use PB to create a kernel with real-time characteristics, you must not use xip technology.
Later, when Windows CE-based products began to use flash, IDE and other permanent storage devices, the file system added a fat. Kernel files and other applications can also be stored on permanent storage devices. The kernel is decompressed by the loader and loaded to the object storage area (object store) of RAM ), all system application files and DLL files contained in the kernel are stored in this area. When an application is executed, the kernel loads the system DLL called by the application to slot 1 (0x0200 109-0x03ff FFFF ). In Windows CE. net, slot 1 is used for xip DLL.
The ram file system is dedicated to object storage. In the previous Article It and ROM file system are Windows CE default file systems. After Windows CE is started, Ram is divided into object store and program memory ). The object storage area uses the RAM File System to save files. It is generally used to save all files unlocked by the kernel. The application memory area is reserved for use when all applications are running. In Windows CE "Control Panel"-"System"-"Memory", you can adjust the ratio of the two storage areas, the slider to the left, then some memory in the object storage area is released and allocated to the application memory area. The slider is opposite to the right.

2. Memory Structure
Windows CE. Net can only manage MB physical memory and 4 GB virtual address space. Different CPU memory management methods are also different. For MIPS and shx series CPUs, physical address ing is completed by the CPU, and the CE kernel can directly access the physical memory of MB. For x86 series and ARM Series CPUs, during kernel startup, it maps all existing physical memory addresses to a virtual address space of 0x8000 or more for later kernel use. OEMs can use oemaddresstable to define the ing between virtual addresses and physical addresses in detail. Oemaddresstable is not a file. It only exists in other files that describe the ing between virtual addresses and actual physical addresses. For example, the file OEM init. ASM contains a piece of code: dd 80000000 H, 0, 04000000 H. It maps a total of 64 MB physical addresses (0x0400 0000 = 64 MB) to virtual addresses from 0x8000 0000 to 0x8400 0000. I will talk about oemaddresstable in a later article about Pb.
The entire 4 GB virtual address space is divided into two parts, from 0x8000 0000 or more for the kernel use, 0x8000 or less for the application use. For details, see the following table:

Address range	Purpose
0x0000 0000 to 0x41ff FFFF	Used by all applications. A total of 33 slots, each of which occupies 32 MB. Slot 0 is used by processes that currently occupy the CPU. Slot 1 is used by xip DLL. Other slots are used by processes. Each process occupies one slot.
0x4200 0000 to 0x7fff FFFF	The region shared by all applications. 32 MB address space sometimes cannot meet the needs of some processes. The process can use the address space in this range. In this area, applications can build heaps, create memory ing files, and allocate large address spaces.
0xa000 0000 to 0 xbfff FFFF	In this range, the kernel repeatedly defines the physical address ing space between 0x8000 0000 and 0x9fff FFFF. The difference is that the virtual address space mapped in this range cannot be used as a buffer. For example, assume that a product has 64 MB physical memory. After oemaddresstable is defined as described above. After the kernel is started, a physical address ing space ranges from 0x8000 0000 to 0x8400 0000. Then, the kernel defines an address space in the same range from 0xa000 0000 to 0xa400 0000, the address space maps 0x8000 0000 to 0x8400 0000 to the same physical address. However, this virtual address space cannot be used as a buffer.
0xc000 0000 to 0xc1ff FFFF	System reserved space
0xc200 0000 to 0xc3ff FFFF	The address space used by the internal program nk.exe.
0xc400 0000 to 0 xdfff FFFF	This range is user-defined static virtual address space, but this address space can only be used for non-buffering. After defining the physical address ing space using oemaddresstable, this range does not change every time the kernel is started, unless the physical memory capacity of the product changes. If the physical memory is increased to MB, the physical address ing space is also doubled. Windows CE. Net also allows users to create static physical address ing spaces. You can call the createstaticmapping function or the nkcreatestaticmapping function to map a physical address to a range between 0xc400 0000 and 0xe000 0000. Note that the Static Virtual Address created using this function can only be accessed by the kernel and cannot be used as a buffer.
0xe000 0000 to 0 xFFFF FFFF	The virtual address used by the kernel. When the kernel requires a large virtual address space, it will be allocated within this range.

Figure 1 Windows CE. Net Memory Structure

3. process address space structure
process address space structure 2. This figure is source to msdn. Windows CE.. NET is different from the previous Windows CE operating system in the process address space. The previous Windows CE also loaded xip DLL into the 32 MB address space of the process, while Windows CE. net loads xip DLL to slot 1 separately, so that the total address space of each process is doubled, that is, 64 MB. I mentioned this when explaining the process.
when an application is started, the kernel selects an idle slot for the program, loads all code and resources, allocates stacks, loads DLL, and so on. When this process obtains the CPU usage right, its entire address space is mapped to slot 0 by the kernel, that is, the address space used by the current process, and then starts running. The address given in the figure is actually a structure mapped to slot 0. It can be seen that the process loads the code segment first. Because each process has a minimum 64kb reserved area, the code segment starts from 0x0001 0000. After the kernel allocates enough virtual address space for the code segment, the allocated space is read-only data and readable/writable data. The allocated space is resource data, and the allocated space is the default heap and stack. Non-xip DLL is loaded starting from the maximum process address down. For non-xip DLL loading, follow the following rules: the kernel first checks whether the DLL to be loaded has been loaded by other processes. If it has been loaded, it will relocate an address. This avoids multiple times of loading the same DLL in the system. If it has not been loaded, find the idle address space in the order from the high address of the slot to the low address of the slot. Allocate enough address space to load the DLL. Because each process is mapped to slot 0 before execution, and all the DLL used by the process may come from different slots ), to avoid address space conflicts between all the used DLL maps to slot 0, the loader of the kernel) when the DLL is loaded, the address of the DLL loaded in all the slots will be searched to ensure that no address conflict will occur when it is mapped to slot 0. If there are two processes in the system, process a loads only dll a, and process B loads dll a and dll B, process B sets aside the address space of DLL, load dll B, that is, when process B is mapped to slot 0, the address space of dll a is adjacent to the address space of dll B and will not conflict. Fortunately, Windows ce dll is very small, and most of the DLL used by an application is the system DLL (which exists in slot 1 ). So currently, the address space of the process is sufficient.

Figure 2 process Address Space Structure

4. Heap and stack
Heap is a continuous virtual address space. Applications can dynamically allocate and release memory blocks in the heap. The advantage of using the heap is that the memory fragmentation is reduced within a certain range. In addition, developers do not need to know the CPU type before allocating memory blocks. Because the page sizes of different CPUs are different, each memory page may be 1 kb, 4 kb, or more. Allocating memory blocks in the heap can be of any size, and directly allocating memory must be measured in memory pages. When an application is started, the kernel allocates a virtual address space of 192kb by default for the process in the address space of the process, but does not submit the physical memory immediately. If kb does not meet the requirements during running, the kernel will find a sufficient free address space in the process address space and then copy the data from the original heap, finally, the address space occupied by the original heap is released. This is because there is a stack in the high address of the default heap, so you must allocate another stack. Windows CE.. Net Heap has obvious disadvantages, different from the heap management in other Windows operating systems, in Windows CE. the memory blocks created in the heap created by. Net cannot be moved. Multiple memory blocks are created and memory blocks are released, in this case, when you need to allocate a larger continuous memory block, the idle memory block is enough, but these memory blocks are separated and do not meet the requirements. Kernels like Windows 2000 or 98 will frequently move distributed memory blocks so that they are together. This is why sometimes a handle is needed instead of a pointer. Due to the disadvantages of Windows CE. Net heap, if developers need to frequently create and release memory blocks in the heap, they 'd better create a separate heap instead of the default heap. I also recommend that you allocate the required address space directly in the global address space (0x4200 0000 to 0x7fff FFFF. The process address space is too small. I will not talk about heap functions here, which is basically the same as other Windows Operating System Heap APIs. See the help documentation.
Stack is also a continuous virtual address space, which is much smaller than the heap space. It is designed for functions. When a function is called (including threads), the kernel will generate a default stack, and the kernel will immediately submit a small amount of physical memory (or prohibit the kernel from committing physical memory immediately ). The stack size is related to the CPU, usually 64 KB, and the top 2 kb is retained to prevent overflow. You can modify the stack size. The specific modification method has already been described when explaining the thread, so we will not repeat it here. Generally, modifying the stack size does not occur. If you modify the stack size when compiling the link, the size of all stacks will change, which is not reasonable. In actual development, it is best not to allocate large and many memory blocks in the stack. If the allocated memory block exceeds the limit of the default stack, the access is invalid and the kernel immediately terminates the process. It is best to allocate a large memory block in the heap of the process and release it before the function returns, or specify the stack size when creating a thread.

5. Memory ing File
Like the virtual memory, the memory ing file is used to reserve an address space and submit it to the physical memory. Early memory ing files do not submit physical memory for callers, but submit file data on permanent memory. Of course, the operating system will reserve a read buffer for permanent memory, so that the reading of file data will be much faster. The features of memory ing files make it suitable for loading EXE or DLL files. This saves memory and reduces the loading time. You can also use it to map large-capacity files, so that you do not have to set a large buffer before reading file data. In addition, memory ing files are often used for inter-process communication and are also the main means of inter-process communication. The communication mechanisms between other processes are implemented based on memory ing files. For faster communication between processes, the current memory ing file can also be submitted to the physical memory, so that the memory ing file can be submitted to both the physical memory and files.
Windows CE. Net also supports anonymous and named Memory ing files. I recommend that you use a memory ing file if you need to read and write large-size files or communicate between threads in different processes during software development, in addition, it is best to allocate IP addresses in the global address space (0x4200 to 0x7fff FFFF. This will make us get twice the result with half the effort.

5.1 map data files
Step 1: Call the createfileformapping function. In Windows CE. net, we recommend that you use this function instead of the createfile function. The createfileformapping function is executed by the kernel and creates a file. It can also open the file created by the createfile function. The parameters are similar to those of createfile. Parameter 1 specifies the file path. Note that the file path format does not contain a drive letter. Parameter 2 specifies the access mode (read or write) and parameter 3 specifies the sharing mode, parameter 4 specifies the Security Attribute (must be set to null), parameter 5 specifies whether to create or open a file, parameter 6 specifies the file attribute, and parameter 7 ignores. For more information about the parameters, see Windows CE. NET Help. The function returns the handle of the created or opened file.
Step 2: Call the createfilemapping function. This function creates an unknown or famous memory ing file object. Parameter 1 is the file handle. This value is returned by the createfileformapping function. Parameter 2 is a security attribute (must be set to null), parameter 3 specifies the protection attribute of the file to be mapped (read-only or read/write ), parameters 4 and 5 are used together to specify the size of the file to be mapped. If the file capacity is too large, the 32-bit integer cannot be expressed. Therefore, the 64-bit variable is used here. Parameter 4 is a high 32-bit value, and parameter 5 is a low 32-bit value. The last parameter specifies the name of the memory ing file. The value can be set to null, indicating that a name is not required.
Step 3: Call the mapviewoffile function. This function is used to reserve a sufficient address space and map the file data on permanent storage to this address space. After ing, this address space is also called a file view. The ing range can be all files or some files. Note that if the file is large, this function will allocate the address space in the global address space. Parameter 1 specifies the handle of the memory ing file object, which is returned by the createfilemapping function. Parameter 2 is similar to parameter 3 in the createfilemapping function and is used to restrict access permissions. Parameters 3 and 4 are used together to specify the starting position of the ing area. Here, parameter 3 is a high 32-digit parameter, and parameter 4 is a low 32-digit parameter. Parameter 5 specifies the size of the ing area. It should be noted that the start position of 64-bit values specified by parameters 3 and 4 may not be a multiple of 64 KB. For other Windows operating systems, the size must be 64 KB. In addition, it should be noted that the help document says that the ing view of a file cannot be consecutive. We recommend that you add a structured exception handling mechanism to prevent unauthorized access. I think this possibility is very small. Generally, for applications with a virtual address space larger than 2 MB, the kernel will be allocated in the global address space. The global address space (0x4200 0000 to 0x7fff FFFF) should be enough space of nearly 1 GB. After all, files in Windows CE are very small. However, adding a structured exception to the code is not a bad thing. We should develop the habit of adding structured exception processing to read and write file data.
Step 4: Perform read/write operations. If the mapviewoffile function is successfully executed, the first address of the ing view is returned. In this case, you can regard the view as a buffer zone and start read or write operations.
Step 5: finish the task. Call the unmapviewoffile function to cancel the file ing view. There is only one parameter, specifying the first address of the view. Then, call the closehandle function to close the memory ing file object. The parameter is the handle. Finally, call the closehandle function again to close the handle of the opened file.

5.2 communication between processes
Sometimes communication is required between processes. The communication mechanisms provided by the system between processes, such as COM and clipboard, use the memory ing file technology at the underlying implementation level. In fact, the idea of communication between processes is very simple. Here I will explain it by the way. In other Windows operating systems, each process occupies 4 GB of address space, 2 GB of address space for the kernel, and 2 GB of address space for the process. All the 2 GB low addresses that a process can access are its own address space. when accessing the kernel address space, it will be restricted by the kernel. Of course, such a process cannot access other processes. To solve the problem of inter-process communication, the memory ing file technology is used as a solution. Originally, the memory ing file only maps files on memory similar to the disk type. For faster communication between processes, the memory ing file can also be submitted to the physical memory. By accessing the same memory ing file object (mapped to the physical memory), two or more processes can access the same physical memory, in this way, other processes can see the data written by a process to the physical memory. While Windows CE only occupies 32 MB of address space for each process, and all processes are in a 4 GB address space, they cannot access each other at will. In Windows CE, in addition to the memory ing file technology, there is also a method that is very suitable for use, that is, the use of Object Storage. The Object Storage Service uses the RAM file system. Composition You can create and read files that exist in the object storage area. The \ Windows directory exists in the object storage area. We can create a file in the \ Windows directory to implement inter-process communication. This method is easy to implement. You only need to call several file API functions and reduce the communication time, because the \ Windows directory is stored in the physical memory, and the data I/O is of course very fast. Using Object Storage Service (OSS) to implement communication between processes is self-developed. msdn or other documents do not provide such instructions. Note the size of the object storage area. In addition, the implementation code is not as good as the memory ing file technology.
The following describes how to use a memory ing file to implement communication between processes. If process a and process B need to communicate, process a needs to first create a memory ing file (you do not have to call the createfileformapping function to create a file before, because you do not need to create a file ). This memory ing file can be in permanent memory or in memory. To reduce the communication time, it is best to submit the physical memory. When process a calls the createfilemapping function, parameter 1 is specified as invalid_handle_value, which indicates that the memory ing file object will submit the physical memory to the address space. You must specify a name for the last parameter. Process B also calls the createfilemapping function, and the parameters are the same. The kernel determines whether a memory ing file object already exists based on the name. If yes, it returns the handle of the original object. Next, you don't have to go into detail. You can execute the command by referring to 5.1. It should be noted that after process B calls the createfilemapping function, it should test the function execution result according to the following code:

 
Handle hmap; hmap = createfilemapping (invalid_handle_value, null, page_readwrite, 0,1000, l "ABC"); If (hmap = NULL | getlasterror ()! = Error_already_exists) {MessageBox (L "Create File mapping fail"); return ;}
 

6. allocate a large virtual address space
you can use a memory ing file to allocate a large virtual address space. You can also directly call the virtualalloc function for allocation. The virtualalloc function is the underlying function for allocating virtual address space. It allocates qualified address space within the calling process and automatically initializes the submitted memory with 0. Pass the first address of the expected virtual address space to parameter 1 (if it is 0, the kernel will automatically find a qualified space). Parameter 2 is the size (in bytes ), parameter 3 is the allocation type (submitted or retained), and parameter 4 is the protection flag (read-only, read/write, execution, etc ). The function returns the first address of the allocated address space. In the process address space, each allocated block has three states: available, retained, and submitted. Parameter 3 indicates the block status. During the experiment, I found that it was unsuccessful to pass a non-0 value to parameter 1. Even if the value 0 was passed to parameter 1 for automatic search by the kernel, the returned value for parameter 1 was not successful again. Release the virtual address space and call the virtualfree function. Virtualfree function parameter 1 specifies the first address, parameter 2 specifies the size, and parameter 3 specifies the release type (uncommit and release ). Returns true if the function is successful, and false if the function fails. Parameter 3 has two tags and cannot be composite. When mem_decommit is specified, the function will cancel the physical memory ing of the virtual address space, but keep this virtual address space. If this virtual address space does not submit the function, it will not fail to return. When the release flag (mem_release) is specified, if the virtual address space contains the same flag (retained or submitted ). The function releases this virtual address space. If a part of the virtual address space has been submitted and other parts have not been submitted, you must first call this function and pass the uncommitted commit flag to cancel the physical memory ing. Call this function again to pass the release flag. In this way, the entire virtual address space can be released. There are other functions related to the virtual address space, such as virtualquery and virtualprotect. This document is not described here. For more information, see Windows CE. NET Help.

note:
process, thread, and memory management: Based on my previous knowledge in the Windows operating system on a PC, I also checked Windows CE. NET Help documentation and the conclusions drawn by technical articles and knowledge base in msdn. Unfortunately, the help document for Windows CE. NET is too simple. I can only combine the knowledge I have mastered with the knowledge I have viewed, and I have also conducted some experiments. I would like to thank all the Windows CE developers who have browsed this article. If you think something is incorrect, I hope to point it out and let me correct the mistake. More people can see accurate articles.