The 14th Week summary of the Design foundation of information security system

9th Chapter Virtual Memory

9.1 Physical and virtual

(1) a system that uses physical addressing : When the CPU executes this load instruction, it produces a valid physical address, which is passed to main memory via the memory bus. The master accesses a 4-byte word starting at physical Address 4 and returns it to CPU,CPU to store it in a register.

(2) a system that uses virtual addressing : When using virtual addressing, the CPU accesses the main memory by generating a virtual address, which is converted to the appropriate physical address before being sent to the memory.

(3) Address Translation : A task that translates a virtual address into a physical address.

Memory Management Unit ( MMU ): Use the query table stored in main memory to dynamically translate the address.

9.2 address Space

(1) address space: an ordered set of non-negative integer addresses:

{0, 1, 2, 3, ...}

Linear address Space : An integer of the address space is contiguous.

Virtual address space : In a system with virtual memory, the CPU is from an N = 2n {0, 1, 2, 3, ..., N-1}

Physical Address space : {0, 1, 2, 3, ..., M-1}

(2) The basic idea of virtual memory : Allow each data object to have multiple independent addresses, each of which is selected from a different address space.

(3) Each byte in main memory has a virtual address selected from the virtual address space and a physical address selected from the Physical address space.

9.3 virtual memory as a tool for caching

(1) A virtual memory (VM) is organized into an array of cells that have n contiguous byte sizes stored on disk. The size of each virtual page is P = 2p (2 P-second side). Physical memory cups are divided into physical pages (PP), and the size is also p-bytes (physical pages are also called page frames)

(2) at any one time, the collection of virtual pages is divided into 3 disjoint subsets :

Unassigned: The VM system also has a page that is not assigned (or created) and does not occupy any disk space

Cached : The allocated pages in the physical memory are currently being deferred.

Not cached : the allocated pages in the physical memory are not being slowed.

(3) The organizational structure of the DRAM cache :

SRAM cache: Represents the L1, L2, and L3 caches that are located between the CPU and main memory.

DRAM cache: Represents the cache of a virtual storage system that caches virtual pages in main memory.

(4) page table :

A page table is an array of page table entries (PTEs). Each PTE consists of a valid bit and an n-bit address field. A valid bit indicates whether the virtual page is slow to exist in DRAM. (Dram is fully connected and any physical page can contain any virtual page).

(5) page hit : When the CPU reads a word from the virtual memory contained in VP2, VP2 is cached in DRAM, and the address translation hardware locates the virtual address as an index PTE2 and reads it from memory. Because a valid bit is set, the address translation hardware will know that the VP2 is cached in memory. So it constructs the physical address of the word using the physical memory address of the Pte.

(6) Missing pages : DRAM cache is not hit.

Exchange or page scheduling: the activity of transferring pages between disk and storage.

On-Demand page scheduling: pages are swapped in from disk (or paged in) DRAM and swapped out (or paged out) disks from DRAM. Wait until the last minute, that is, when there is a miss, the strategy of changing into the page.

VM pages (before): a reference to a word in VP3 is not hit, triggering a missing pages

VM page Fault (after): The pages fault handler chooses VP4 as a sacrifice and replaces it with a copy of VP3 from disk. After the fault-pages handler restarts the instruction that caused the missing pages, the instruction reads the word normally from memory without any further exception.

(7)箥: The size of the working set exceeds the size of the physical memory.

9.4 virtual memory as a tool for memory management

(1) VMS how to provide a separate address space for the process . The operating system maintains a separate page table for each process in the system: (Multiple virtual pages can be mapped to the same shared physical page)

In this example, the page table of process I maps VP1 to PP2,VP2 mapping to PP7. Similarly, the page table of process J maps VP1 to PP7,VP2 maps to PP 10.

(2) VMS simplify linking and loading, code and data sharing, and storage allocation for applications.

9.5 virtual memory as a tool for memory protection

The SUP bit indicates whether the process must run in kernel (superuser) mode to access the page. A process running in kernel mode can access any page, but a process running in user mode only allows access to those pages where the SUP is 0.

Such as: Process I runs in user mode, then it has read VP 0 and VP 1 permissions. However, it does not allow access to VP 2.

9.6 Address Translation

(1) page hit : Steps for CPU hardware execution

The first step: The processor generates a virtual address and transmits it to the MMU.

Step two: MMU generates a PTE address and obtains it from the cache/main memory request.

Step three: Cache/main Memory returns PTE to the MMU.

The fourth step, the MMU constructs the physical address and transmits it to the cache/main memory.

Fifth step: Cache/Main Memory returns the requested data Word to the processor.

(2) physical pages : requires hardware and operating system cores to complete:

The first step: The processor generates a virtual address and transmits it to the MMU.

Step two: MMU generates a PTE address and obtains it from the cache/main memory request.

Step three: Cache/main Memory returns PTE to the MMU.

Fourth step: The valid bit in PTE is 0, so the MMU triggers an exception, passing the control of the CPU to the fault handler in the operating system kernel.

Fifth step: The page fault handler determines the sacrifice page in the physical memory, and if it has been modified, swap it out to disk.

Sixth step: The page is paged into the new page and updates the PTEs in the memory.

Seventh step: The fault-pages handler returns to the original process and executes the instruction that caused the missing pages again. The CPU sends the virtual address that caused the missing pages back to the MMU.

(3) TLB ( translation fallback buffer ): there is t = 2t (2 of the T-Square), then the TLB index (TLBI) is composed of the lowest bit of the VPN, and the TLB tag (TLBT) is made up of the remaining bits in the VPN.

(4) TLB hit :

First step: CPU generates a virtual address

Step Two and step three: the MMU extracts the corresponding PTE from the TLB.

Fourth step: MMU translates this virtual address into a physical address and sends it to cache/main memory.

Fifth step: Cache/Main Memory returns the requested data Word to the CPU.

When the TLB misses, the MMU must remove the corresponding PTE from the L1 cache (the newly-taken PTE is placed in the TLB and may overwrite an existing entry).

(5) Multilevel Page Table :

Each PTE in the first-page table maps a 4MB slice in the virtual address, where each piece is made up of 1024 contiguous pages. For example, PTE0 maps the first piece, PTE1 maps the next piece, and so on. If the address space is 4gb,1024 a PTE is sufficient to cover the true entire space.

(6) synthesis: End-to-end address translation : A---> B---> C

Assume:

The memory is addressed by Byte.

Memory access is for 1-byte words.

The virtual address is 14 bits long (n = 14).

The physical address is 12 bits long (M = 12).

The page size is 64 bytes (P = 64).

9.8 Memory Mapping

(1) definition : Linux has initialized the contents of this virtual memory area by associating a virtual memory area with an object on disk.

(2) a virtual memory area can be mapped to one of two types of objects :

A.unix file System: A zone can be mapped to a contiguous portion of a regular disk file.

B. Anonymous files: An area can also be mapped to an anonymous file, the anonymous file is created by the kernel and contains all binary zeros. Because there is no actual data transfer between the disks and the storage, the pages in the area that are mapped to anonymous files are sometimes called binary zero pages.

(3) Shared Objects

An object can be mapped to an area of the virtual storage, either as a shared object or as a private object. If a process maps a shared object to a region of its virtual address space, then any operation of the process on that zone is not visible to other processes that also map the shared object to their virtual storage. Also, these changes are reflected in the original objects on disk.

On the other hand, a change to a zone mapped to a private object is not visible to other processes, and any operations done by the process-year-old area are not reflected in the disk's objects. A virtual storage area that is mapped to a shared object is also called a shared zone.

A. Process 1            maps an object to an area of its virtual storage; B. Process 2 after mapping to the same shared object;

(4) a private write-time copy object:

A. Two processes have mapped the private write-time copy objects:

As a illustrates a situation where two processes map a private object to different areas of their virtual storage, but share the same physical copy of the object. For each process that maps private objects, the page table entries for the corresponding private zone are marked as read-only, and the zone structure is marked as private write-time copies. As long as no process master writes its own private area, they can continue to share a separate copy of the objects in the physical memory. However, as long as a process tries to write a page in the private zone, the write operation triggers a protection failure.

B. After process 2 wrote a page of the private zone:

When the fault handler notices that the protection exception is caused by the process attempting to write a page in the private copy area of the write, it creates a new copy of the page in the physical memory, updates the page table entry to point to the new copy, and then restores the writable permissions for the page, such as B-map. When the fault handler returns, the CPU performs the write operation again, and the write operation is now performed on the newly created page.

(5) Execve function:

Assume a program that runs in the current process:

Execve ("a.out", NULL, NULL);

The following steps are required to load and run A.out:

L Delete a user area that already exists.

L Map Private areas. Creates a new structure for the text, data, BSS, and stack areas of the new program.

L Map shared areas.

L Set the program counter.

(6) User-level memory mappings using the Mmap function:

The MMAP function requires the kernel to create a new virtual memory area, preferably an area starting from address start, and map a contiguous slice of the object that the file describes FD to to this new region. The size of the continuous object slice is the length byte, starting at the offset byte from the beginning of the file.

(7) Themunmap function deletes the area of the virtual storage :

#include <unistd.h>

#include <sys/mman.h>

int Munmap (void *start, size_t length);

The Munmap function removes the area from the virtual address start, which consists of the next length byte. The next reference to the deleted zone will result in a segment error.

9.9 Dynamic Storage Allocation

(1) Classification of distributors:

L Explicit Allocator : requires an application to explicitly release any allocated block. The C program allocates a block by calling the malloc function and frees a block by calling the free function.

L Implicit allocator : Requires the adapter to detect when an allocated block is no longer in use by the program, then release the block. An implicit allocator is also called a garbage collector, and the process of automatically releasing unused allocated blocks is called a trash collection.

(2) malloc and FREE functions:

#include <stdlib.h>

void *malloc (size_t size);

The malloc function returns a pointer to a memory block of at least size bytes, which is aligned for any data object type that might be contained within the block.

Shows how the malloc and free functions manage a 16-word small heap of C programs. Each box represents a 4-byte word. The rectangle that is marked with a thick line corresponds to the allocated block (shaded) and the free Block (no shadow). Initially, the heap consists of a 16-word, double-word-aligned, free block.

A. The program requests a 4-word block. The response of malloc is to cut out a 4-word block from the front of the free block and return a pointer to the first word of the block.

B. The program requests a block of 5 words. The response of malloc is to allocate a 6-word block from the front of the free block. In this example, malloc fills an extra word in the block to keep the free block aligned with the double-word boundary.

C. The program requests a block of 6 words. And malloc cuts out a 6-word block from the front of the free block.

D. The program releases a block of 6 words in Figure B. Note: After the call to free returns, the pointer P2 still points to the freed block. It should be the responsibility to no longer use P2 until it is reinitialized by a new malloc call.

E. The program requests a 2-word block. In this case, malloc allocates part of the block that was freed in the previous step and returns a pointer to the new block.

(3) Why use dynamic memory allocation : Often they know the size of certain data structures until the program is actually running.

(4) Requirements and objectives of the dispenser

A. Requirements :

L process any request sequence. An app can have arbitrary allocation requests and free request sequences, as long as the constraints are met: Each release request must correspond to a currently allocated order, which is obtained by a previous allocator request.

L respond to requests immediately. Allocators are not allowed to reorder or buffer requests in order to improve performance.

L Use only heaps: in order to make the allocator extensible.

l to aligning (alignment requirements): You can save any type of data object.

l Do not modify allocated blocks

B. Target :

1. Maximum throughput rate: The number of requests completed per unit of time. Rationality can mean the worst run time of a woman's allocation request is linearly related to the number of free blocks, and the run time of a release request is a constant.
2. Maximizing memory Utilization: Given the order of n allocation and release requests:

R0, R1, R2, ...., RN

If an application has a P-byte block, then the resulting allocated payload is P bytes. After the request RK is complete, the aggregation payload is expressed as PK, which is the sum of the payload of the currently allocated block, and HK represents the current size of the heap.

UK = Maxi<k PI/HK

The purpose of the allocator is to maximize the peak utilization Un-1 throughout the sequence.

(5) Fragments

1. category :

Internal fragmentation: Occurs when an allocated block is larger than the payload.

Quantization of the internal fragment = (Allocated block size-payload size). At any given moment, the number of internal fragments depends only on the pattern that was previously requested and how the allocator was implemented.

External fragmentation: When the free memory is aggregated enough to satisfy an allocation request, there is not a single free block large enough to handle this request to occur.

External fragmentation is much more difficult than the quantification of internal fragments, because it depends not only on the implementation of the previous request pattern and allocator, but also on the pattern of future requests.

(6) implicit idle link list :

1. A simple heap block pattern :

A block consists of a word's head, payload, and possibly some extra padding. The head encodes the size of the block (including the head and all the padding), and whether the block is allocated or idle. If we impose a double-word constraint, then the block size is always a multiple of 8, and the minimum bit size of 3 bits is always zero. Therefore, we only need to store 29 highs of the size, releasing the remaining low 3 bits to encode additional information.

For example, suppose we have an allocated block with a size of (0x18) bytes, then its head is: 0x00000018 | 0x1 = 0x00000019

If a block size of 0x28 bytes of free block has the following header:

0x00000028 | 0x0 = 0x00000028

The heap is organized into a queue of contiguous allocated blocks and free blocks:

(7) merge of marked with boundary

1. the previous and subsequent blocks are assigned

2. The previous block is allocated, and the back block is idle

3. The front block is idle, and the following block is allocated

4. the back block and the front block are free

When we try to merge the current block and the back block in the memory, as long as the previous block is idle, it will need to use its foot, if we put the allocated/idle bits of the front block in the current block of the extra low, then the allocated block will not need the foot, So we can use the extra space as payload.

(8) an explicit idle link list

L FIFO (Flfo): Maintains the list of newly freed blocks at the beginning of the list. Releasing a block can be done within a constant time, and merging is

L Maintain linked lists in order of address, where each block in the linked list is less than its successor address

disadvantage : the free block must be large enough to contain all the required pointers, as well as the head and possible feet. This leads to a smaller block size and potentially increases the level of internal fragmentation.

(9) detached list of idle links

1) simple separation of storage : The free list of each size class contains blocks of equal size, and the size of each block is the size of the largest element in the size class.

2) separation adaptation :

3) Partner system : A special case of separation adaptation, where each size class is a power of 2. The basic idea is to assume a heap size of 2m words, we maintain a separate free block for each block size 2k, where 0≤k≤m. The request block size is rounded up to the nearest power of 2. Initially, there is only one free block with a size of 2m characters.

Key Benefits: Quick Search and Quick Merge

Major drawback: requiring a power of block size 2 can lead to significant internal fragmentation.

9.10 Garbage Collection

1. garbage collector : A dynamic storage allocator that automatically frees allocated blocks that are no longer needed by the program. The process of automatically reclaiming heap storage is called garbage collection . In a system that supports garbage collection, applications explicitly allocate heap blocks, but they are never explicitly freed.
2. garbage collector Basics: The garbage collector will be treated as a forward-to-reach graph. Node is divided into a set of root nodes and a set of heap nodes. Each heap node corresponds to an allocated block in the pair.

When an arbitrary root node is present and reaches the valid path of P, we say that node P is reachable. The unreachable node corresponds to garbage and is not used again by the application

The 14th Week summary of the Design foundation of information security system