MIT6.828 Jos System LAB2

MIT6.828 lab2:http://pdos.csail.mit.edu/6.828/2014/labs/lab2/

LAB2 is mainly about the paging process of the system, there is a simple virtual address to the physical address of the process. About the system paging, the MIT6.828 virtual address into the physical address--level two paging: http://blog.csdn.net/fang92/article/details/47320747.

The following are mainly lab2 of several exercise of the problem-solving process.

1. First Boot_alloc () function:

static void *
Boot_alloc (uint32_t N)
{
	static char *nextfree;	Virtual address of next byte of free memory
	char *result;

	if (!nextfree) {
		extern char end[];<span style= "White-space:pre" >	</span>//end points to the end of T He kernel ' s BSS segment:
		Nextfree = ROUNDUP ((char *) end, pgsize);
	}
	if (n==0)
		return nextfree;
	result = Nextfree;
	Nextfree + = n;
	Nextfree = ROUNDUP ((char*) Nextfree, pgsize);
	return result;
}

Boot_alloc (unit32_t N) is primarily a n-byte address space, returning the first address of the application space. If n is 0, the Nextfree (the first address of unallocated space) is returned. Where the assigned address is page-aligned, or 4K aligned.

This function, which cleverly applies uninitialized global variables and static variables, is automatically initialized to the 0 feature. function to first define the static local variable nextfree, which points to the first address of the free memory space. Because it is uninitialized, the variable is automatically initialized to 0, so the first time the Boot_alloc () function is called, the value of Nextfree is 0, and the following statement is executed:

if (!nextfree) {
		//end points to the end of the kernel ' s BSS segment
		Nextfree = ROUNDUP ((char *) end, pgsize); 
  
   }
  

End points to the end of the BSS segment, and you can see the information for each segment of the program by calling Objdump-h kernel.

As you can see from the picture above, the position of the. BSS section is the bottom of the program, note that. The comment segment is a comment message for some programs and is not in memory. So end uninitialized global variable in the. BSS segment. So end is pointing to the last address of the entire memory. (What's going on in the BSS section).

The next roundup is a macro definition that is used for 4K alignment, and then returns an address that points to the first address of the next free page of end.

So the first time the system calls Boot_alloc () This function, the first nextfree will be pointed to the first free page of the first address. Next, the address is assigned according to the N entered. If n=0, returns the Nextfree, otherwise assigns an n-byte address, which returns the first address of the assigned address. Note that 4K alignment is required throughout the process.

2.mem_init () function

This function only needs to be partially replenished. The main point is to request a certain address space for the struct PageInfo's pointer pages. First look at the definition of struct PageInfo:

struct PageInfo 
	{ 	//next page on the free list.
		struct PageInfo *pp_link;
		uint16_t pp_ref;
		}

This structure is primarily used to hold information about all physical pages in memory. Each of the pageinfo corresponds to a physical page.

PageInfo has two main variables:

Pp_link indicates that the next free page, if pp_link=0, indicates that the page is assigned, otherwise, the page is not allocated and is a free page.

Pp_ref represents the number of pages that are referenced, and if 0, indicates that the page is free. (this variable is similar to the reference count of pointers in smart pointers).

The supplementary code is simple, that is, a bit of pages requesting enough space (Npages pages) to hold these structures and initialize them with memset:

	Pages = Boot_alloc (npages * sizeof (struct PageInfo));
	memset (pages, 0, npages*sizeof (struct PageInfo));

About pages:

Pages is a structure used to manage physical memory pages. First, pages is the No. 0 page of physical memory for the entire system, and the value of pages is a virtual address. Because the structure is continuous, so through pages[i]-pages can get the page number I, in the i<<12 can get pages[i] the corresponding page of the physical memory, because the implementation of the system's physical memory and virtual memory conversion is relatively simple, virtual memory = physical Memory + 0xf0000000. So through the structure of pages, when you know a specific physical page, you can easily get physical pages corresponding to the physical address and virtual address, as shown in figure.

3.page_init ()

The Page_Init () function is basically the initialization of the structure pages, that is, when the system is just beginning to run, the physical memory allocation to initialize the operation, according to the above prompts step by step write can be, relatively simple.

void
Page_Init (void)
{
	
	size_t i;
	for (i = 0; i < npages; i++) {
		if (i = = 0)
			{	pages[i].pp_ref = 1;
				Pages[i].pp_link = NULL;
			}
		else if (i>=1 && i<npages_basemem)
		{
			pages[i].pp_ref = 0;
			Pages[i].pp_link = page_free_list; 
			Page_free_list = &pages[i];
		}
		else if (i>=iophysmem/pgsize && i< extphysmem/pgsize)
		{
			pages[i].pp_ref = 1;
			Pages[i].pp_link = NULL;
		}
	
		else if (i >= extphysmem/pgsize && 
				i < ((int) (Boot_alloc (0))-kernbase)/pgsize)
		{
			pages[i ].pp_ref = 1;
			Pages[i].pp_link =null;
		}
		else
		{
			pages[i].pp_ref = 0;
			Pages[i].pp_link = page_free_list;
			Page_free_list = &pages[i];
		}

	}

In this initialization function, there is one thing to be aware of. The entire pages structure uses Pp_link to differentiate whether a page is a free page. For non-free pages, only need to put Pp_ref, Pp_link 0 can be, non-free pages do not have to pass something to make him interrelated. However, for the free page, due to the need to apply the page in real time, after the non-free page release page, you also need to add the corresponding page to the structure of the free page, so the structure design of the free page is important. The structure of the free page is designed to meet: when the application page, can quickly get free pages; 2. When the page is released, the page can be inserted quickly.

In this experimental system, a relatively ingenious structure is adopted, which is used to complete the application and invocation of the page by page_free_list this variable.

First, the structure of the page is shown in the following figure:

As you can see from the graph, the structure of the entire free page is a bit like an inverted list, because the structure of the free page is built from bottom to top, and the head node is a null pointer in the list. While using the process, the entire structure is similar to a stack, using the advanced post-out principle, page_free_list point to the stack vertex.

At first, Page_free_list =0,page_free_list represents the current idle page of the system. When initializing, insert the first free page table, assign the page_free_list to the pp_link of the Free page table, and update the page_free_list so that it points to the first free page table (page0 in the diagram), and then operations until the free page table is inserted. When Free_page, simply follow the above procedure and insert the free page that is released. When the system requests a free page table, simply give the page table that page_free_list points to, and then update page_free_list. When page_free_list = null, there is no free page table.

Overall, the use of the entire free page is more like a stack. In the whole structure, page_free_list plays a key role.

This structure, for larger unit space management is more effective, that is, simple and efficient. If you reduce the size of the unit page, consider the extreme situation, to 16B as a page, then this page management method is not. As you can see, there are two variables in each pageinfo, a bit pointer, an int, a page size of 8 bytes, 8 bytes to manage 16-byte pages, and then the entire memory, 50% are the pages structure. The space wasted is huge. For the 4K-sized pages in the experiment, the space utilization is very high, and the entire page occupies approximately 8/4000 of the space.

In general, this management approach and structure is used for large unit memory management. For the management of heap memory, because the heap request is in 4 or 8 bytes, the equivalent of the page size bit 4b/8b, this management method is not OK.

4.page_alloc () and Page_free ()

Page_alloc () is a page application function, the whole function of the simple idea is to read and update page_free_list to apply for the page. Just pay attention to the situation where the free page has been applied.

struct PageInfo *
page_alloc (int alloc_flags)
{	if (page_free_list = = null)
		return null;

	struct pageinfo* page = page_free_list;
	Page_free_list = page->pp_link;
	Page->pp_link = 0;
	if (Alloc_flags & Alloc_zero)
		memset (Page2kva (page), 0, pgsize);
	return page;
}

Page_free is to release the page, but also relatively simple, only need to pay attention to Pp_ref and Pp_link is 0.

void
page_free (struct PageInfo *pp)
{
	
	if (pp->pp_link! = 0  | | pp->pp_ref! = 0)
		Panic ("Page_ Free isn't Right ");
	Pp->pp_link = page_free_list;
	Page_free_list = pp;
	return; 
}

Part 2:virtual Memory At first, let people see sections and pagination related content, as well as some of the translation and page translation of some things, before doing the following things, be sure to understand the pagination related things, segmented because here temporarily involved in not much, can be seen later: Look at Chapters 5 and 6 of the Intel 80386 Reference Manual Address: http://pdos.csail.mit.edu/6.828/2014/readings/i386/toc.htm (segmented content, have time to see more) write a simple blog: MIT6.828 Virtual address to physical address--level two page: http://blog.csdn.net/fang92/article/details/47320747 in the experiment, it is important to point out that, in general, the C language pointer, the address it points to is the offset of the segment, That is, offset, is not a virtual address, but also need to add a segment base to get the virtual address of the pointer variable, and then through the page translation, get the actual physical address. The following sentence is important for the function of the complement experiment: from code executing to the CPU, once we ' re in protected mode (which we entered first thing in boot/ Boot. s), there's no-directly use a linear or physical address. all memory references is interpreted as Virtua L addresses and translated by the MMU, which means all pointers in C is virtual addresses. This sentence tells us that all the addresses in the program are virtual addresses, and the system will translate the physical address through the MMU. In other words, all the addresses inside the program are virtual addresses. Even if it is a physical address, the program calls it as a virtual address and translates it into a physical address. This point to pay special attention, at the beginning, did not notice, resulting in the writing function later there are some serious problems, it is difficult to find ... In the Jos inside, in the lab2 inside can temporarily not pipe segment translation, can be regarded as the whole system is a paragraph, namely its section base is 0, paragraph range (limit) is 0xFFFFFFFF. The system is divided into a segment, so the system can be seen asPure page-paged form. In the paging, the conversion of the address is also relatively simple, Jos is designed to 0-256m physical address mapping to the beginning of the 0xf0000000 256M virtual address, so from the physical address to convert virtual address easier. Question: This wood is to be divided into the Jos system, what is the physical address, what is the virtual address. Because the system will often have to carry out the related operations of the address, so often to be forced to convert, a unsigned int variable into an address, or vice versa, so in the program, you need to distinguish between the two. The problem of the above topic is relatively simple, x should be uintptr_t, in the program, any pointer is a virtual address (segment offset).
One of the following exercises is page management, primarily the management of page mappings.

A total of 5 functions:

1. Pgdir_walk (): Returns the address (PTE) of the corresponding Level two page table for VA

2. Boot_map_region: [VA, va+size) mapped to [PA, pa+size)

3.page_lookup: Returns the virtual address VA corresponding to the physical address of the Pages page page

4.page_remove: Unmap the VA and its corresponding page

5. Page_insert (): Map VA to the specified physical page table page

1. Pgdir_walk (): Returns the address (PTE) of the corresponding Level two page table for VA

The main function of this function is to give a virtual address VA and Pgdir (the first address of Page Director table) and return the PTEs corresponding to the VA (page table entry). When the VA corresponding level two page table exists, only need to directly follow the page translation process to give the address of the PTE can be. However, when the VA corresponding Level two page table has not been created, it is necessary to manually request the page and create the page. The process is relatively simple, but when the address of the PTE is returned at the end, it is necessary to return the virtual address corresponding to the PTE address, instead of directly giving the physical address of the Pte. Because the program can only execute virtual address, the physical address given will also be treated as a virtual address, the general will cause a segment error.

pte_t *
pgdir_walk (pde_t *pgdir, const void *va, int create)
{
	//Fill this function in
	int pdeindex = (UN signed int) VA >>22;
	if (pgdir[pdeindex] = = 0 && Create = = 0)
		return NULL;
	if (pgdir[pdeindex] = = 0) {
		struct pageinfo* page = page_alloc (1);
		if (page = = null)
			return null;
		page->pp_ref++;
		pte_t pgaddress = PAGE2PA (page);
		Pgaddress |= Pte_u;
		Pgaddress |= pte_p;
		Pgaddress |= Pte_w;
		Pgdir[pdeindex] = pgaddress;
	}
	pte_t pgadd = Pgdir[pdeindex];
	Pgadd = pgadd>>12<<12;
	int pteindex = (pte_t) VA >>12 & 0x3ff;
	pte_t * pte = (pte_t*) Pgadd + pteindex;
	Return Kaddr ((pte_t) Pte); <span style= "White-space:pre" >	</span>//Be sure to return the virtual address, and the physical address will throw an error.
}	

2. Boot_map_region: [VA, va+size) mapped to [PA, pa+size)

This function is relatively simple, the function is to map [VA, va+size] to [PA, pa+size)

static void
boot_map_region (pde_t *pgdir, uintptr_t va, size_t size, physaddr_t pa, int perm)
{while
	
	(size) c3/>{
		pte_t* pte = Pgdir_walk (Pgdir, (void*) VA, 1);
		if (pte = = NULL)
			return;
		*pte= PA |perm| pte_p;
		
		Size-= pgsize;
		PA  + = pgsize;
		VA  + = pgsize;
	}
}

3.page_lookup: Returns the virtual address VA corresponding to the physical address of the Pages page page

struct PageInfo *
page_lookup (pde_t *pgdir, void *va, pte_t **pte_store)
{	pte_t* pte = Pgdir_walk (Pgdir, VA, 0);
	if (pte = = null)
		return null;
	pte_t pa =  *pte>>12<<12;
	if (Pte_store! = 0)
		*pte_store = Pte;
	Return Pa2page (PA);	
}

4.page_remove: Unmap the VA and its corresponding page

The corresponding page of VA is released, in other words, the mapping of VA is canceled. Note here that after the page is released, you need to clear 0 of the values stored in the PTEs corresponding to the VA, otherwise when querying VA corresponding PTEs, errors will occur, the system will mistakenly think VA and pa or there is a correspondence relationship.

void
Page_remove (pde_t *pgdir, void *va)
{	pte_t* pte;
	struct pageinfo* page = page_lookup (Pgdir, VA, &pte);
	if (page = = 0)
		return;
	*pte = 0;
	page->pp_ref--;
	if (page->pp_ref ==0)
		page_free (page);
	Tlb_invalidate (Pgdir, VA);
}

Over here

void
tlb_invalidate (pde_t *pgdir, void *va)
{
	//Flush The entry only if we ' re modifying the current addres s space.
	For now, there are only one address space and so always invalidate.
	INVLPG (VA);
}

About TLB, you can see: http://blog.chinaunix.net/uid-16361381-id-3044981.htm

Because the TLB is a cache of page table entries, because of this, we are temporarily not considering the problem of multiple processes in the system. So you can assume that the cache entry stored in the TLB is a process, that is, the address space is one. So when the VA is unmapped, it is necessary to check the cache page entries in the TLB that have no VA, and if so, do the corresponding operations.

5. Page_insert (): Map VA to the specified physical page table page

There are three things to consider in this function:

1.va does not have a corresponding mapping page

The 2:VA has a corresponding map page, but is not the specified page

3.: VA has the corresponding mapping page and is the same as the specified page.

For Case 1, the simplest, directly to the VA and page mapping can be, the specific method is to find the VA corresponding to the PTEs, and then modify the value inside the PTE, so that it is the corresponding page of the physical address.

For Case 2, the corresponding page of the VA should be released (REMOVE,REF-1), and then the same processing method as in case 1.

For Case 3, be aware that when two page is the same, you cannot return directly. Because the Page_insert function does not only have to map the virtual address and the page, it also sets the privileges of the page (Pte_u,pte_p,pte_w), and if the original page's privileges are different from the current privileges, then return directly, there will be a problem.

int
Page_insert (pde_t *pgdir, struct PageInfo *pp, void *va, int perm)
{

	pte_t* pte = Pgdir_walk (Pgdir, VA, 1) ;
	if (pte = = NULL)
		return-e_no_mem;
	if ((Pte[0] &  ~0xfff) = = PAGE2PA (PP))
		pp->pp_ref--;
	else if (*pte! = 0)
		page_remove (Pgdir, VA);

	*pte = (PAGE2PA (PP) & ~0xfff) | Perm | pte_p;
	pp->pp_ref++;
	return 0;
}

Now for the Part2 section of the page management to make a summary:

Part2 inside the various functions is actually the above level two page directory to do a management, mainly the VA and PA mapping.

The Pgdir_walk () function is the address of the PTE that returns the two-level page table for VA, and when querying the first-level page directory, the VA does not have an address mapping if the corresponding PTE's physical base address is not stored in the PDE. At this point, depending on the creat, you decide whether you need to create a level two page. If necessary, apply for a free page, as a Level two page table of VA, and return the address of the corresponding Level two page table (Pte address).

The Boot_map_region () function is to fill in the corresponding physical address in the PTEs corresponding to the VA to complete the mapping of the virtual address to the physical address.

The Page_lookup () function is the page that returns the corresponding physical address of the VA.

The Page_remove () function cancels the mapping of the VA and the corresponding page, that is, the contents of the PTEs corresponding to the VA are zeroed, and the corresponding page and VA lose the mapping, so the pp_ref-1 is required.

The Page_insert () function is a mapping relationship between the VA and the requested page, which is to populate the PTE with the corresponding physical address of the page.

In general, Part2 is writing various functions that use the creation of level two pages.

Part 3:kernel Address space part3 is primarily a user-space and kernel-space thing. The main distinction is ulim, in the Ulim above is the kernel space, in the following section is the user space. As shown in figure:

In Part2, the first-level page directory and the two-level page table have a low 12 of the address, there are privileged bits of the page.

The above figure is the address distribution in Pte and PDE, since the low 12 bits are offset and are determined by VA, so the low 12 bits are used as the flag bits of the page. Here the main 3-bit, that is, the U,W,P three flag bit.

P: Represents whether the page is valid, if 1, indicates the page is valid. Otherwise, the page is invalid, the page cannot be mapped, or an error occurs.

W: Indicates whether the page is writable. If 1, the page can be write operations, otherwise, the page is read-only page, can not be modified.

U: Indicates whether the user program can use the page. If bit 1 indicates that this page is a user page, the user program can use and access the page. If 0, the user program cannot access the page, only the kernel can access the page.

The above page flag bit, can effectively protect the security of the system. Since the operating system runs in kernel space (with the exception of microkernel, some of its system functions are performed in user-state), the general user program is run on the user space. So the user program of the crash, will not affect the operating system, because the user program does not have permission to modify the contents of the kernel address. This effectively isolates the operating system and user programs, and strengthens the stability of the system.

In the middle of the entire address part of the address section [Utop,ulim], the user program and the kernel can be accessed, but this part of the memory is read-only, can not be modified, in this part of memory, the main storage kernel of some read-only data. Some of the tables that may be GDT are present in this part of the address space.

In the next low memory, it is to the user program to use the address.

There are 3 functions left:

The first: the page and virtual address upages where the pages structure resides are required to map each other. As long as the size of the pages structure is calculated, it can be mapped by Page_insert ().

<span style= "White-space:pre" >	</span>int perm = Pte_u | pte_p;
	int i=0;
	 n = ROUNDUP (npages*sizeof (struct PageInfo), pgsize);
	for (i=0; i<n; i= i+pgsize)
		Page_insert (Kern_pgdir, Pa2page (paddr (pages) + i), (void *) (upages +i), perm);

The second one: mapping the virtual address [kstacktop-kstksize, Kstacktop] to the physical address starting with Bootstack (Bootstack is actually storing its virtual address, need to convert the bit physical address), which is the static mapping between the addresses. So use the boot_map_region () to do it.

	Perm =0;
	Perm = Pte_p | Pte_w;
	Boot_map_region (Kern_pgdir, Kstacktop-kstksize, ROUNDUP (Kstksize, Pgsize), Paddr (Bootstack), perm);

The third is also a static mapping of an address, which maps the address from [Kernbase, 2^32) to [0, 2^32-kernbase]. The only trouble here is that the 32-bit machine has no way to express 2^32, to get a size by some means.

	int size = ~0;
	size = Size-kernbase +1;
	Size = ROUNDUP (size, pgsize);
	Perm = 0;
	Perm = Pte_p | Pte_w;
	Boot_map_region (Kern_pgdir, kernbase, size, 0, Perm);

The basic part of the whole lab2 is finished, feel nothing, but do a long time ...

Next comes the last few question:

We have placed the kernel and user environment in the same address space. Why would user programs not being able to read or write the kernel ' s memory? What specific mechanisms protect the kernel memory?

As said above, mainly rely on Pte_u to protect.

What is the maximum amount of physical memory, this operating system can support? Why?

Because in memory, upages a total of 4 m of memory to store pages, that is, a total of 4m/8byte=0.5m pages, the total memory size is 0.5m*4k=2g, so total 2G memory maximum.

How much space overhead was there for managing memory, if we actually had the maximum amount of physical memory? How are this overhead broken down?

2G memory, the total number of pages is 0.5M, pages structure (PAGEINFO) is the size of 0.5m*8byte=4m,page director is 4K, because the number of pages is 0.5M, so page table is 0.5m* 4BYGE=2M, so the total is 6m+4k

Generally speaking, the 32-bit system can put the maximum memory should be 4G. The main reason for this calculation is that the Upages memory range is only 4 m, and if Upages expands to 8 m, then the system can be addressed to 4G at the limit.