Memory management mechanism for deep SQL SERVER 2000 (ii)

Server
Memory management mechanism for deep SQL SERVER 2000 (ii)





Http://msdn.microsoft.com/data/default.aspx?pull=/library/en-us/dnsqldev/html/sqldev_01262004.asp



Access to large address applications (Large-address-aware executables)

Before Windows adds support for/3GB parameters, An application is unable to access a pointer with a high setting. A 32-bit pointer only has the first 31-bit address space that can be accessed by user-mode applications. The remaining one is not. So there are some smart developers who are unwilling to waste this bit on the memory address space for other purposes. (For example: You can use to flag a pointer to a data type assigned by another application). This/3gb the parameter to encounter a problem, because this type of program cannot conveniently distinguish a valid pointer reference to the memory space above the 2G dividing line with a memory address space below 2G, But its high position has been used as a pointer to it. Basically, if a machine starts with a/3GB parameter, the application cannot run. To address this situation, Microsoft has added a new flag bit to the WIN32 PE file characteristics field to indicate whether a program is running in a (Large-address-aware executables) mode that has access to a large address. When the flag bit is used (Image_ File_large_address_aware), the executable header characteristics the 32-bit position of the field. The application can hint at a bit of the header of this executable file that Windows can handle pointers with high settings. This will not result in any abnormal results. When the flag is set and the windows that support this method is started by/3GB, the operating system provides an extended private user-mode address space to the application. You can use tools such as: DumpBin and ImageCfg (Dumps the header of an executable file) to examine the flag bit of an executable file. Visual C + + uses the/LARGEADDRESSAWARE connector switch to indicate that Image_file_large_address_aware,sql SERVR has activated the flag bit, so you can support the/3GB parameter in Windows This parameter is used in the version so that the operating system automatically expands the user memory space of SQL Server.

Note: Windows checks the IMAGE_FILE_LARGE_ADDRESS_AWARE flag bit when the executable is started, but ignores the DLL file. dll code must automatically handle pointers to high position bits.



Physical Memory Extension

Intel processors since Pentium Pro started, future processors support a memory-mapped mode of physical memory extension (PAE). PAE mode provides the physical memory space that can access 64GB. In PAE mode, the memory snap-in (Memory Management Unit (MMU)) is still executing: Page catalog entry page directory entries (PDEs) and Page table entry page tables entries (PTEs), But on top of that there is a new level: page catalog pointer table. Also, in PAE mode PDEs and PTEs are 64-bit (not just standard 32-bit), so the system can map more memory space than the standard conversions because the PDEs and PTEs modes are twice times the width of the address. This is not just an addition to the page Catalog pointer table. The page Catalog pointer table is used to manage these large-capacity tables and indexes. Some special versions of the Windows kernel need to run in PAE mode. The kernel is integrated in Windows 2000 and later versions and is embodied in Ntkrnlpa.exe files in a single-processor machine, and is embodied in Ntkrnlpamp.exe files in multiprocessor machines. You can activate PAE mode by adding the/PAE parameter to the Boot.ini file with the addition of/3GB and/USERVA parameters.



Address Window Extensions

The Windows Address window extension (AWE) mechanism allows applications to access more than 4GB of physical memory. A 32-bit pointer is an integer that can only hold a value of 0x00000000 to 0xFFFFFFFF, that is, a linear memory address space within 4GB may be referenced, and AWE allows an application to bypass these restrictions and access all memory space through the operating system.

In concept, AWE is not a new thing. At the beginning of computer development, operating systems and applications have used similar mechanisms to circumvent pointer constraints. For example: We go backwards to the DOS age, and the 32-bit extension is often used to allow a 16-bit program to access his own memory address space. Some special purpose managers and APIs often use expanded memory and extended memory. You may remember such a long time ago product quarterdeck QEMM-386 often used to do such a thing. A typical mechanism is to allow a pointer to access a space beyond its own limits (for example, an address is too large to be in its own pointer) to pass a pointer between a window or a region in an accessible address space and a memory address that is inaccessible to itself. How AWE works: You can provide an area (window) in an accessible address space as a segmented transmission area to transfer memory addresses that cannot be accessed in the user's memory space.

To use AWE, an application requires:

1. The allocated physical memory address can be accessed through the Allocateuserphysicalpages API function. This function requires the caller to have the lock Pages in memory permission.

2. Establishes an area in the memory space that can be accessed. The VirtualAlloc API function can be used as a mapping window for mapping a physical memory.

3. Complete the mapping of physical and virtual memory through the Mapuserphysicalpages or Mapuserphysicalpagesscatter WIN32 API functions.

AWE already exists in all Windows 2000 and later operating systems, even in operating systems with less than 2GB of physical memory, and the most typical application is on machines with 2GB or more physical memory, because this is the only way a 32-bit processor can access 3GB of the following memory space. If you activate AWE support in a SQL Server system that is below 3GB of physical memory, the system ignores this option and converts it to the virtual Memory manager instead. An interesting feature of AWE memory is the fact that data is never exchanged with disk. You may notice that the unique AWE API program references the memory space that can be accessed as physical memory. This is exactly the following: AWE memory is the physical memory space that does not interact with the system's virtual paging file.

The Virtual memory window is used to cache the request of AWE read and write access to physical memory. Therefore, when you configure this window to be page_readwrite the only feature that can be protected is to marry the VirtualAlloc API function. Don't be surprised, it also means you can't use the VirtualProtect API function to protect the memory area from modification and access.

Note: There are no dedicated tools to investigate application AWE memory usage (Task Manager, Performance Monitor and monitoring system, and so on), showing the number of AWE memory usage per program. This does not include a trajectory for each program using the number of AWE memory, which is not included in the working memory set of each program.



Comparison of/3GB and awe

The ability to increase the user program address space almost 50% of applications are adjusted through memory, which is a very fast and popular tool in the Windows Memory management mechanism. And the Windows AWE memory mechanism is also very flexible and stable. As I said earlier, when you add 1G of user memory space, the memory is obtained by reducing the core memory space (from 2G to 1G). Because the core code is running for the entire memory space is very narrow a piece even for 2G of space, shrinking these memory space means that the internal core architecture will shrink. The most important of these is the table that Windows uses to manage physical memory, and when you shrink the core memory space to 1G, you limit the size of the table so that you can manage only the maximum 16GB of physical memory. For example, if your application is running on a server with 64GB of physical memory and you add/3GB parameters at startup. You can only access 25% of the total memory space-the rest of the 48GB of memory is inaccessible to the operating system or the application. AWE allows you to access a higher memory space than the/3GB parameter. Obviously, your parameters for/3GB only add 1GB of user memory space, and these increased memory space is only valid for those applications that are automatically obtained by large addresses, but only 1GB. In contrast to the/3GB parameter, AWE can make the entire physical memory effective for the operating system and for applications that use the AWE WIN32 API. As a result, AWE is more complex to use and operate, and more flexible and scalable.

Not that/3GB is better than awe-but it does exist, for example: if you need to allocate more memory space, but not in awe (thread stack, memory lock, Process plan), you may find that/3GB back better.