Information Security System Design Foundation 14th Week study Summary

Information Security System Design Foundation 14th Week study Summary

"Learning Time: 7 hours"

"Learning content: chapter9--virtual Memory"

First, the content of the textbook Understanding (Personal Understanding section with "" marked) 1. Virtual Storage (VM)

• function :
  1. The main memory is considered as a cache of the address space stored on disk, only the active area is protected in main memory, and the data is transmitted back and forth between the disk and main memory as needed;
  2. simplifies memory management by providing a consistent address space for each process;
  3. Protects the address space of each process from being destroyed by other processes

"Understanding: In fact 2. With 3. is closely related. Because the virtual memory provides a consistent representation of the address space for each process, it is like providing a uniform room numbering mechanism for each process so that the processes do not go into the wrong room and cannot find a room in the case of a room. "

• Status :
  1. Virtual memory is central: it is hardware exception, hardware address translation, main memory, disk file and kernel software interaction Center;
  2. Virtual memory is powerful: it can create and destroy memory slices, can map memory slices to a portion of a disk, and so on;
  3. Virtual memory is dangerous if it is not properly operated

2. Physical and Virtual addressing

• Physical Addressing (PA): Each byte in main memory has a unique physical address, which is called physical addressing;
• Virtual Addressing (VA): The CPU generates a virtual address and then accesses the main memory with this address, which is converted to the appropriate physical address before it is sent to the memory (this process is called address translation)

3. Basic idea of virtual memory

Each data object is allowed to have multiple independent addresses, each of which is selected from a different address space (for example, virtual address space or physical address space).

4. How does virtual storage work as a caching tool?

• Idea:
  • First, the VM is organized into an array of cells of n contiguous byte sizes stored on disk. Each byte has a unique virtual address, which is used as an index to the array.
  • Second, the VM system splits the virtual memory into a fixed-size virtual page, with each virtual page having a size of p=2^p bytes; Similarly, the physical memory is split into a physical page (also called a page frame) with a size of p bytes

"Understanding: That is, the virtual memory of the space used to store the real address of the data (so that virtual memory equivalent to the index), a virtual storage cells can be organized to point to the only corresponding physical page"

• Virtual Memory Page Collection
  • Unassigned: The VM system does not associate them with any data and takes up no disk space;
  • Cached: The allocated pages in the physical memory are currently being slowed;
  • Not cached: No more allocated pages exist in physical memory

5. Page Table

• Effect: Maps a virtual page to a physical page. Each time the address translation hardware translates a virtual address into a physical address, the page table is read. The operating system is responsible for maintaining the contents of the page table.
• Structure: A page table is an array of page table entries (PTEs); Each page in the virtual address space has a PTE at a fixed offset in the page table. For our purposes, we assume that each PTE consists of a valid bit and an n-bit address field. A valid bit indicates the starting position of the physical page in which the virtual page is cached.

6. Missing pages

• DRAM cache misses are called missing pages.
• Processing process:
  • The page fault is called by the kernel to call the fault handler, the program will select a sacrifice, and swap it out of memory;
  • The kernel copies the required entries from the disk to the location before the page is sacrificed, and then returns;
  • When the exception handler returns, it restarts the instruction that caused the missing pages, which re-sends the virtual address that caused the missing pages to the address translation hardware;
  • At this point, the page hits
• Concept Supplement:
  • In the customary parlance of memory, blocks are called pages;
  • The activity of transferring pages between disk and storage is called switching or paging;
  • The page is swapped into DRAM and swapped out of the disk from the DRAM, and has been waiting until a hit occurs before swapping in the page; This strategy is called on-demand page scheduling

7. Address Translation

• Explanation: Formally speaking, address translation is a mapping between an element in the virtual address space (VAS) of an n element and the Physical address space (PAS) of an M element.
• CPU, the page table base register points to the current page table;
• n bit includes the following two parts: a virtual page offset for a P-bit and a virtual page number for a (n-p) bit;
• mmu generates a PTE address and requests it from cache/main memory,
• cache/main memory returns PTE to the MMU;
• mmu constructs the physical address and passes it to cache/main memory;
• cache/Main memory returns the requested data Word to the processor
• CPU execution steps (missing pages)
  1. The processor generates a virtual address and passes it to the MMU;
  2. The MMU generates a PTE address and requests it from cache/main memory;
  3. Cache/Main Memory returns PTE to the MMU;
  4. The significant bit in PTEs is 0, triggering an exception, passing the control of the CPU to the fault handling program in the operating system kernel;
  5. The page fault handler determines the sacrifice page in the physical memory, and if it has been modified, swap it out to disk;
  6. The page fault handler is paged into the new pages and updates the PTEs in the memory;
  7. The fault-pages handler returns to the original process and executes the instruction that caused the missing pages again. The CPU will resend the virtual address that caused the page fault to the MMU, because the virtual pages are now slow to exist in the physical memory, so a hit occurs.

8.linux Virtual Memory System

• Linux maintains a separate virtual address space for each process, where the kernel virtual memory resides on the user stack;
• Kernel virtual memory contains code and data structures in the kernel, and some are mapped to a contiguous set of physical pages (primarily for convenient access to specific locations, such as where I need to perform I/O operations)
• Linux organizes virtual storage into a collection of areas (also called segments). An area is a continuous slice of the already existing (allocated) virtual memory;
  • Meaning: Allow the virtual address space to have gaps, the kernel does not have to record those pages that do not exist, such pages do not occupy memory;
  • Regional structure
    • Vm_start: Point at the beginning of the area;
    • Vm_end: Point at the end of the area;
    • Vm_prot: Describes the read and Write permission permissions for all pages contained within this area;
    • Vm_fags: Describes whether the pages in this area are shared with other processes, whether the process is private, and so on;
    • Vm_next: The next structure that points to the list

9. Memory Mapping

• Concept: Linux (and some other forms of Unix) initializes the contents of this virtual memory region by associating a virtual memory area with an object on a disk;
• Type:
  • Common Files in UNIX file systems: An area can be mapped to a contiguous portion of a regular disk file, such as an executable target file;
  • Anonymous files: An area can also be mapped to an anonymous file, the anonymous file is created by the kernel, which contains all binary 0 (the first time the CPU references a virtual page in such a region, the kernel will find the appropriate sacrifice page in the physical memory and then overwrite it with binary zeros)

10. Comparison of shared objects & private objects

"What is the difference between a shared object and a private object, or a process, which is inseparable from the virtual memory?"

• Introduction: An object can be mapped to an area of the virtual storage, either as a shared object or as a private object.
  • If a process maps a shared object to a region of its virtual address space, then any write to the zone by the process is also visible to other processes that also map the shared object to its own virtual address space "regardless of how many shared areas the object is mapped to, Only one copy of the object is needed in the physical memory ";
  • Changes made to a zone mapped to a private object are not visible to other processes, and any writes that the process makes to the zone are not reflected in the objects on disk

(Fig. 1)

• Depth:
  • Private objects are mapped to virtual memory through clever techniques of copy-on-write, and for each process that maps private objects, page table entries for the corresponding private areas are marked as read-only, and the zone structure is marked as a write-time copy of the private object;
  • As long as no process attempts to write its own private area, they can continue to share;
  • Whenever a process tries to write a page in a private area, the write action triggers a protection failure, and when the fault handler notices that the protection exception is caused by the process trying to write a page in the private copy area of the write, it creates a new copy of the page in the physical storage area, The Update page table entry points to this new copy and then restores its writable permissions

"The benefit of this is that the physical memory space is protected to the maximum and maximum amount of time, so that it will never be wasted when it is not necessary."

11. Garbage Collection

• Concept: The garbage collector is a dynamic storage allocator that automatically frees allocated blocks that are no longer needed by the program. These blocks are called garbage. The process of automatic recycling is called garbage collection. The garbage collector periodically identifies the garbage blocks and calls free accordingly, putting the blocks back into the idle list

Second, after-school exercise screening 1. Exercises 9.2

Determine the number of PTEs required for the combination of the following virtual address size (n) and page size (P)

n = 16,p = 4k--> #PTE = 16;

n = 16,p = 8k--> #PTE = 8;

n = 32,p = 4k--> #PTE =1m

"Resolution: Because each virtual page is P = 2^p bytes, there is a total of 2^n/2^p = 2^ (n-p) possible pages in the system, each of which requires a page table entry (PTE)"

2. Exercises 9.3

Given a 32-bit virtual address space and a 24-bit physical address, for the following page size p, determine the number of bits of Vpn,vpo,ppn,ppo

"Supplement: vpn--virtual page number; vpo--virtual page offset; ppn--physical page number; ppo--physical page offset"

"Understanding: We have 32 virtual address bits and 24 physical address bits, and the page size is 1KB, which means that for VPO and PPO, 10 bits are required (to differentiate the 2^10 specific units within each page); then the rest is Vpn,ppn"

P = 1kb-->vpn = 22,vpo = 10,ppn = 14,ppo = 10

P = 4kb-->vpn = 20,vpo = 12,ppn = 12,ppo = 12

3. Exercises 9.5

Write a C language program MMAPCOPY.C, use mmap to copy any size of the disk file to stdout, the input file name must be passed as a command line parameter

"The Answer:"

#include "csapp.h"void mmapcopy(int fd,int size){    char *bufp;    bufp = Mmap(NULL,size,PROT_READ,MAP_PRIVATE,fd,0);    Write(1,bufp,size);    return;}int main(int argc,char **argv){    struct stat stat;    int fd;    if(argc !=2)//因为传进来的参数应该有两个，一个是文件名（题目要求用命令行传递）；第二个是    {        printf("usage:%s ,<filename>\n",argv[0]);        exit(0);    }    fd = Open(argv[0],O_RDONLY,0);    fstat(fd,&stat);    mmapcopy(fd,stat.st_size);    exit(0);}

"Parse:"

• struct STAT stat: reference http://blog.csdn.net/dumgeewang/article/details/7741033 stat structure content is as follows:

St_mode file permissions and file type information

St_ino the inode associated with the file

St_dev the device that saved the file

St_uid file owner's UID number

St_gid file belongs to the master GID number

The time that the St_atime file was last visited

St_ctime file permissions, owner, group, or content last modified time

The content of the Stmtime file was last modified at the time. (andThe difference between St CTime is obvious)

St_nlink the number of hard connections on this file

• open (argv[0],o_rdonly,0) : Reference/HTTP// blog.csdn.net/liuyang1990i/article/details/7596630

  • function prototype: int open (const char *path, int oflags,mode_t mode);

  • function Description: Open is generally used to create files, or to open or create files, you can set the properties of the file and the user's permissions and various parameters.
• fstat: Reference http://blog.csdn.net/dumgeewang/article/details/7741033 define function int fstat (int fildes,struct stat * BUF), which is used to copy the state of the file referred to by the parameter fildes to the structure referred to in the parameter buf (struct stat).
• Mmap: Requires the kernel to create a new virtual storage area and map a contiguous slice of the object that the file descriptor FD refers to in this region

"Description:"

In this way, the idea of a function is understandable:

The two parameters entered by the user are executed in turn (this is not important, it is cat or other) and the absolute path to the file (for example, c:/cpp/a.cpp);

In the main function, first determine whether the number of inputs is valid (illegal exit), and then the parameter 2 points to the file opened and return descriptor FD; using Fstat function to obtain its file state store stat structure, jump to mmapcopy function;

The Mmapcopy function calls Mmap to the virtual address space, and then uses the Write function to write the contents of the virtual space (that is, the disk file contents) mapped to the stdout (the file descriptor is 1)

Third, experience

The content of this chapter has a meaning of "point" for the contents of many previous chapters (such as stacks, such as processes), because it is necessary to understand the virtual memory when it involves the process, the mutex and the shared content, so reading this chapter is more enlightened.

Information Security System Design Foundation 14th Week study Summary