SQL Server 2000 Memory Management Insider

Introduction

In this column, we will explore the SQL Server memory Management Insider from the developer's perspective. That is, we'll discuss how SQL Server uses APIs and operating system features to manage memory and how it works. Exploring a product in this way will help us to understand the ideas of the product developers and the methods of use they have designed. Understanding the working principle of a product and its design purpose is the key to mastering this product.

We'll start with some basic basics of Windows memory management. Like all 32-bit Windows applications, SQL Server uses the Windows Memory Management feature to allocate, release, and manage memory resources. As with all other Windows applications, SQL Server invokes the WIN32 memory management API function to interact with the memory resources provided by the operating system.

Because almost all memory allocations in SQL Server use virtual memory (not the memory heap), the vast majority of the memory allocation code is ultimately accomplished by invoking the Win32 VirtualAlloc or the VirtualFree function. SQL Server calls VirtualAlloc to reserve, commit the allocated virtual memory, and invoke VirtualFree to free virtual memory.

Virtual Memory vs Physical Memory

On the x86 series processor, Windows provides a 4GB virtual memory workspace for all processes. The word "virtual" means that this memory is not the usual memory, it is just an address range and is not associated with the physical storage unit. These address spaces are used when the process requests memory allocations and are associated with specific physical storage units. However, these physical storage units are not necessarily physical memory, it is usually disk space, or rather, the paging file of the operating system (System paging files). This is why multiple applications can run concurrently on a system of 128M memory, each with a 4GB virtual memory address space-it is not real memory, but it can be understood as memory for the application. Windows transparently handles data copies of paging files so that applications can use more memory than the actual physical memory of the machine and enable the application to fairly access the machine's physical memory.

This 4GB address space is divided into two parts: the user mode section and the Kernal mode section. By default, the size of each section is 2GB, and on Windows NT-series operating systems, you can change this default setting through the switches in Boot.ini (Windows NT, Windows XP, Windows xp-based, and Windows Server 2003 belongs to the Windows NT family, Windows 9x and Windows Me do not belong.

Figure 1:windows The virtual address space of the process into user mode (application) and Kernal mode (operating system) two parts

Each application has its own virtual memory address space, but the operating system and device drivers share the same private address space. Each virtual memory page (memory page) is associated with a specific processor mode (processor mode), and the processor must work in the required mode in order to access a virtual memory page. This means that the application cannot directly access virtual memory in Kernal mode, and the system must switch to kernal mode to access kernal mode's memory space.

Application Memory Tuning

The 3GB startup option (available in Windows 2000 Advanced Server and Datacenter and subsequent versions of Windows) allows you to change the default size of these two address space sections. It allows the process's user mode address space to be extended from 2GB to 3GB, with the corresponding cost of kernal mode's address space being reduced from 2GB to 1GB. In Windows parlance, this feature is called Application Memory tuning or 4GB tuning (4GT). You can enable application application Memory tuning by adding the/3GB switch in the [operating Systems] section of the Boot.ini file. In general, by setting the [operating Systems] portion of the Boot.ini file, the system is configured to be 3GB or not 3GB-enabled, so that you can make a selection when the system starts.

Warning: You can also use the/3GB switch on Windows Professional and Windows Server, and the negative result is that the kernal mode is reduced to 1GB, but does not add to user mode space. In other words, you've reduced the space in kernal mode without getting any benefit.

Note: Windows XP and Windows Server 2003 introduce a new startup option/userva, which can be better controlled than using/3GB alone, with/3GB. When you add/3GB to boot.ini, you can add/userva,/userva at the same time as the advantage of using/3GB alone is that it allows you to specify an exact address space size value for user mode access. For example,/userva=2560 configures 2.5G of space for user Mdoe, and the remaining 1.5G is used for kernal mode. The warning message above is also applicable when using the/USERVA option.

Large-address-aware Executables

Before/3GB supports joining Windows, applications cannot use the highest bit of pointers, and User mode applications can only access the address space represented by the first 31 bits of a 32-bit pointer. For the remaining 1, some smart developers do not want to waste the 1 bits in the process space and use it for other purposes, such as pointers that identify the application-specific type of address assignment. This poses a challenge after the introduction of/3GB because this type of application cannot distinguish between pointers that refer to more than 2GB of memory, and those that refer to the memory below 2GB but are set at a high level for other reasons. Basically, using/3GB to start the machine can crash such an application. To address this problem, Microsoft's characteristics field in the Win32 PE file format (which defines the format of executable EXE and DLL structure under Windows) is supported by a new identity bit that indicates whether the application supports large addressing capabilities. Set the 32nd bit of the characteristics field in the executable header to enable the Image_file_large_address_aware identity bit. By setting this identity bit of the application header, the application can handle those pointers that are set at the highest bit, not because this bit brings any more meaning. When this identity bit is set, starting with the/3GB option on the correct version of Windows, the system provides a private extended user mode address space for the process. You can use DUMPBIN, imagecfg, and so on to analyze the executable header tool to see if the application is enabled for this identity bit. Visual C + + provides support for image_file_large_address_aware through the/largeaddressaware connection switch. SQL Server has this identity bit enabled, so when you start using the/3GB switch on the correct version of Windows, the SQL Server's user mode address space is extended.

Note: Windows checks the Image_file_large_address_aware identity when the process starts, ignoring the identity of the DLL. For those pointers with the highest bit set, the DLL code must be handled correctly.