Virtual Memory Address (translation)

Virtual Memory and Memory Mapping

Virtual Memory was introduced in 1959. the main goal was to significantly simplify programming by Using acting the quirks of physical memory's hierarchy. nowadays, virtual memory management is an essential component in almost every operating system's kernel. today, I discuss the main concepts of virtual memory and its advantages.

Virtual Memory was introduced in 1959. by abstracting the essence of physical memory layers, programming is simpler. Now, virtual memory management is required on each operating system kernel.

Real addressing and virtual addressing

Real and virtual addresses

In the olden days, a process wocould access physical memory directly. This model, knownReal addressing, Is still in use in embedded systems and some PDAs. It's also used in DOS. However, real addressing mode is problematic for several reasons:

In the ancient chaotic age, processes can directly access physical memory. This mode is called a real address, which is still used in embedded systems and some PDAs; it is also used in DOS. However, the actual address model may have problems for the following reasons:

• System stability. A process may accidentally tread on another process's memory. this can occur, for instance, when a process tries to write to an uninitialized pointer whose value belongs to the address space of a different process. worse yet, a process cocould overwrite critical kernel code, thus freezing the system.
• System stability. A process may use the memory of other processes. For example, a process writes something to an uninitialized pointer address, but this address belongs to another process. It is even possible that the process operates the key kernel code, causing the system to crash.
• Security. Apart from accidental trespassing, real addressing enables a process to steal and modify other processes 'memory.
• Security. In addition to the above reasons, the real address mode allows a process to modify the memory of other processes
• Performance. in real mode addressing, swapping and memory mapping (I'll explain these techniques shortly) are usually unsupported. consequently, the system isn' t scalable. crashes and freezes are common annoyances in such systems.
• Performance. In real address mode, switching and memory ing are usually not supported, and the system cannot be expanded. Crashes and crashes are common in such systems.

How virtual addressing works

In virtual addressing mode, every program is allocated a private address space that is completely isolated from other processes 'address spaces. For example, if address0x4000aef6In process a points to a certain variable, the same address in process B may point to some other data variable or it may not exist in B. a virtual memory address is, therefore,Alias; The physical location to which it points is hidden from the process and from the programmer. For example, address0x4000aef6Can be mapped to the physical address0x102788ffIn the system's Ram, or to offset 0 from cluster 1740 on the system's hard disk. under this model, each process sees the system's memory as if it were the sole process running on the machine.

In virtual address mode, each program restricts the address space, and the address space between processes is isolated. For example, if 0x4000aef6 in process a points to a variable, 0x4000aef6 in process B may point to other data variables, or the address does not exist in process B. Therefore, the virtual memory address is an alias; the physical location it points to is invisible to the process and program personnel. For example, the address 0x4000aef6 can be mapped to the physical address of RAM.0x102788ff or the 1740th cluster of the disk. In this model, each process deems itself in control of all the memory of the system.

Early virtual memory managers used kernel modules to translate a virtual address to a physical address and vice versa. Today, this mapping is saved by special hardware, so it's completely transparent.

Early virtual memory management used the system module to convert virtual addresses and real addresses. Currently, this type of ing is completely transparent through special hardware.

Virtual addressing enables the kernel to load and unload memory dynamically. A frequently-used code section can be stored in the ram, whereas other sections of the program can be mapped to a swap file. when the process accesses a memory address that is mapped to a swap file, the kernel reloads that portion to the system's Ram. from a user's point of view, there's no difference whatsoever between the two (could t performance, of course ).

The virtual address enables the kernel to dynamically load/unload memory. The code that is frequently used in the program can be stored in Ram, and other parts should be mapped to the swap file. When process access is mapped to the memory address in the swap file, the kernel reloads this part to ram. From the user's point of view, there is no difference between loading in Ram and when it is used.

Memory Mapping

Memory mapping is a common concept in POSIX and Windows systems. it enables a process to map disk files to its address space, treating them as memory blocks rather than files. usually, the mapping consists of translating a disk location to a virtual address and vice versa; there is no copying of disk data into the ram. memory mapping has several uses:

Memory ing is a common concept in windows and POSIX systems. It allows the process to map disk files to the address space of the process itself and process the files as memory blocks rather than files. Usually, ing is to convert the disk location and virtual memory address. During the ing, no disk data is copied to ram. Memory ing has the following functions:

• Dynamic Loading. By mapping executable files and shared libraries into its address space, a program can load and unload executable code sections dynamically.
• Dynamic loading. By ing executable files and shared libraries to address spaces, the program can dynamically load/unload executable code parts
• Fast File I/O. When you call file I/O functions, suchRead ()AndWrite (), The data is copied to a kernel's intermediary buffer before it is transferred to the physical file or the process. this intermediary buffering is slow and expensive. memory Mapping eliminates this intermediary buffering, thereby improving performance significantly.
• Fast I/O. When I/O functions are called, data is first copied to the temporary cache of the kernel before being transmitted to the process. The temporary cache is small and slow. The memory ing does not use the temporary cache. Therefore, the program performance is greatly improved.
• Streamlining file access. Once you map a file to a memory region, you access it via pointers, just as you wocould access ordinary variables and objects.
• Streamline file access. Once the file is mapped to the memory area, you can access it through pointers, just like using common variables or objects.
• Memory persistence. Memory Mapping enables processes to share memory sections that persist independently of the lifetime of a certain process.
• Memory continuity. Memory ing allows the process to share the life cycle without relying on the memory area of a specific process.

The POSIX<Sys/Mman. h>Header Des memory mapping syscalland data structures. Because this interface is more intuitive and simpler than that of windows, I base my memory mapping example on the POSIX library.

TheMMAP ()System call:

caddr_t mmap(caddress_t map_addr,        size_t length,        int protection,        int flags,        int fd,        off_t offset);

 

Let's examine what each parameter means.

Map_addrIs the address to which the memory shocould be mapped.NullValue allows the kernel to pick any address (normally you 'd use this value ).LengthContains the number of bytes to map from the file.ProtectionIndicates the types of access allowed to the mapped region:

1 PROT_READ //the mapped region may be read2 PROT_WRITE //the mapped region may be written3 PROT_EXEC //the mapped region may be executed

FlagsContains various mapping attributes; for instance,Map_lockedGuarantees that the mapped region is never swapped.

FDIs the mapped file's descriptor.

Finally,OffsetSpecifies the position in the file from which mapping shoshould begin, with offset 0 indicating the file's beginning.

In the following example, the program maps the first 4 kb of a file passed in command line into its memory and then readsIntValue from it:

 1 #include <errno.h> 2 #include <fcntl.h> 3 #include <sys/mman.h> 4 #include <sys/types.h> 5  6  int main(int argc, char *argv[]) 7  { 8  int fd; 9  void * pregion;10  if (fd= open(argv[1], O_RDONLY) <0)11  {12  perror("failed on open");13  return –1;14  }15  /*map first 4 kilobytes of fd*/16  pregion=mmap(NULL, 4096, PROT_READ,MAP_SHARED,fd,0);17  if (pregion==(caddr_t)-1)18  {19  perror("mmap failed")20  return –1;21  }22  close(fd); //close the physical file because we don‘t need it23  //access mapped memory; read the first int in the mapped file24  int val= *((int*) pregion);25 }

ToUnmapA mapped region, useMunmap ()Function:

int munmap(caddr_t addr, int length);

ADDRIs the address of the region being unmapped.LengthSpecifies how much of the memory shocould be unmapped (You may unmap a portion of a previusly-mapped region ). the following example unmaps the first kilobyte of the previusly-mapped file. the remaining three kb still remain mapped to the process's ram after this call:

munmap(pregion, 1024);

Summary

Without virtual addressing, programming wocould be much more difficult than you 'd imagine. furthermore, define common programming concepts such as dynamic memory allocation and multiple processing wocould be harder to implement. virtual addressing thus proves once more that "there's no problem that can't be solved by an additional level of indirection."

Dynamic Memory Allocation and virtual memory

Every application running on your operating system has its unique address space, which it sees as a continuous block of memory. in fact the memory is not physically continuous (it is fragmented), this is just the impression the operating system gives to every program and it's called virtual memory. the size of thevirtual memory is the maximum size of the maximum size your computer canaddress using Pointers (usuallyon a 32-bit processor each process can address 4 GB of memory ). the natural question that arises is what happens when a process wants to access more memory than your machine physically has available as Ram? Due to having a virtual address space, parts of the hard disk can be mapped together with real memory and the process doesn't have to know anything about whether the address isphysically stored in Ram or on the hard disk. the operating system maintains a table, where virtual addresses are mapped with their correspondent physical addresses, which is used whenever arequest is made to read or write to a memory address.

Typically, in each process, the virtual memory available to that process iscalled itsAddress Space. Each process's address space is typically organized in 6 sections that are already strated in the next picture: Environment Section-used to store environment variables and command linearguments; the stack, used to store memory for function arguments, return values, and automatic variables; the heap (Free store) used for Dynamic Allocation, twodata sections (for initialized and uninitialized static and global variables) and a text section where the actual code iskept.

The heap

To understand why the dynamic memoryallocation is time consuming let's take a closer look at what is actually happening. the memory area where new gets its blocks of memory for allocation (usually called free store or heap) is wrongly strated in the following picture:


When new is invoked, it starts looking for a free memory block that fits the size for your request. supposing that such a block of memory is found, it is marked as reserved and a pointer to that location is returned. there are several algorithms to accomplish this because a compromise has to be made between scanning the whole memory for finding the smallest free block bigger than the size of your object, or returning the first one where the memory needed fits. in order to improve the speed of getting a block of memory, the free and reserved areas of memory are maintained in a data structure similar to binary trees called a heap. the various algorithms for finding free memory are beyond the scope of this article and you can find a thorough discussion about them in D. knuth's monograph The Art of computer programming -- Vol.1, fundamental algorithms ). this overhead combined with the risk for memory leaks makes the use of automatic memory (allocated on the stack) preferred whenever possible and the allocation is not large.

How much virtual memory do you get

Even though every application has its own 4 GB (on 32-bit systems) of virtualmemory, that does not necessarily mean that your program can actually use all of thatmemory. for example, on Windows, the upper 2 GB of that memory are allocatedto the operating system kernel, and are unavailable to the process. (therefore, any pointer starting with 0x8xxxxxxx is unavailable in user space .) on Linux, the upper 1 GB iskernel address space. typically, operating systems provide means forchanging these defaults (such as the/3 GB switch on Windows. it is rare, however, that you really want or need to do so.

Address Space fragmentation

Another concern with memory allocation is that if you allocate memory innon-contiguous blocks, over time "holes" will develop. for example, if youallocate 10 KB and it is taken from the middle of a 20 mb chunk of memory, then you can no longer allocate that 20 mb a one chunk of memory. doing thisenough times will cause you to no longer be able to allocate 20 mb at once. this can cause allocation failures even when there is free memory. note thatthis is true even with virtual memory because what matters is that you need acontinuous block of addresses, not a continuous block of physical memory.

One way to address this problem is to avoid doing things that haveproblems due to fragmentation, such as avoiding largeallocations -- anything more than a few tens of MB is certainly asking fortrouble. second, when heap implementations help you with this already byallocating a large chunk of virtual address space and carving it up for you (usually the heap allocates address space from the operating system and then provides smaller chunks when requested ). but if you know that you will have a class that has a lot of small instances, you cocould overload operatornew and preallocate a large continuous chunk of memory, splitting offsmall pieces for each class from that chunk.

Virtual Memory Address (translation)