2017-2018-1 20179202 "Linux kernel Fundamentals and analysis" Eighth week assignment

One, the load of the executable program 1. preprocessing, compiling, linking

gcc –e –o hello.cpp hello.c   //预处理gcc -x cpp-output -S -o hello.s hello.cpp //编译   gcc -x assembler -c hello.s -o hello.o-m32  //汇编gcc -o hello hello.o   //链接成可执行文件，使用共享库

The gcc -o hello.static hello.o -static statically compiled hello.static put the required items in the C library into the executable file. With ls –l the command, you can see that hello only 7k,hello.static has about 700K.

2. elf Files

The ELF (excutable and linking format) is the standard for a file format. By readelf-h hello looking at the executable file Hello's head (-a View all information,-H only to see the header information), the head is marked with the target file type ELF32. Entry Point address is the entry for the program at 0X8048310,
You can execute the first line of code address that the file is loaded into memory to begin execution. After the head there are some code data and so on. The format of the executable file and the address space of the process have a mapping relationship that loads the code snippet and data segment of the Elf file into the address space of the process when the program is to be loaded into memory to run.

There are three object files in the elf file: the relocatable (relocatable) file holds the code and the appropriate data, Used with other object files to create an executable file or a shared file (mostly. o files); executable (executable) file holds a program to execute, which indicates that exec (Ba_os) How to create a program process image (how the operating system loads the executable file and where to start it); The shared object file holds the code and the appropriate data to be linked by two linker. The first is the link editor (static link), which can be used with other relocatable and shared object files to create an additional object. The second is a dynamic linker that unites an executable file and other shared object files to create a process image.

3. Dynamic linking

Dynamic links have two ways to perform dynamic linking (most) and runtime dynamic chaining when loading.

(1) Shared library

The Sharedlibapi () function is defined in shlibexample.h, and shlibexample.c is the implementation of this function. gcc -shared shlibexample.c -o libshlibexample.so -m32generate. So files with ' (plus-32 when executing in 64-bit environments). This generates a shared library file.

#include <stdio.h>#include "shlibexample.h"int SharedLibApi(){    printf("This is a shared libary!\n");    return SUCCESS;}

(2) Load shared library dynamically

Dllibexample.h defines the Dynamicalloadinglibapi () function, dllibexample.c is the implementation of this function. Also used to gcc -shared dllibexample.c -o libdllibexample.so get dynamically loaded shared libraries.

#include <stdio.h>#include "dllibexample.h"#define SUCCESS 0#define FAILURE (-1)int DynamicalLoadingLibApi(){    printf("This is a Dynamical Loading libary!\n");    return SUCCESS;}

(3) The main function uses two kinds of dynamic link libraries.

  #include <stdio.h> #include "shlibexample.h" #include <dlfcn.h>int main () {printf ("This is a main    Program!\n ");    /* Use Shared Lib */printf ("Calling Sharedlibapi () function of libshlibexample.so!\n"); Sharedlibapi (); Call the shared library directly/* Use dynamical Loading Lib */void * handle = Dlopen ("libdllibexample.so", rtld_now);//Open the dynamic library and load it into memory I        F (handle = = NULL) {printf ("Open Lib libdllibexample.so error:%s\n", Dlerror ());    return FAILURE;    } int (*func) (void);    char * ERROR;    Func = Dlsym (handle, "Dynamicalloadinglibapi");        if (Error = Dlerror ()) = NULL) {printf ("Dynamicalloadinglibapi not found:%s\n", error);    return FAILURE;    } printf ("Calling Dynamicalloadinglibapi () function of libdllibexample.so!\n");      Func (); Dlclose (handle); Unload library return SUCCESS;}  

You can see that the main function only has shlibexample (shared library), no include dllibexample (dynamic load shared library), but include DLFCN. The shared library can be called directly because the interface file for the shared library is previously added. However, if you want to invoke the dynamic load shared library, use the Dlopen defined in Dlfcn.h.

To gcc main.c -o main -L/path/to/your/dir -lshlibexample -ldl -m32 generate the executable file. Note that this only provides shlibexample-L, and does not provide information about Dllibexample, but indicates-LDL. -DL dynamic loading, compile MAIN.C, no relevant information is specified, but within the program. The experiment is as follows:

3. Code Analysis

The current executable program executes, executes to the execve of the time into the kernel state, with the Execve loaded executables to the current process to cover the execution of the program, when the EXECVE system call returned, has returned is not the original executable program, Is the starting point for the new executable program (main function). The shell environment executes the EXECVE, loading command-line arguments and environment variables, and system call calls Sys_execve when it is trapped inside the kernel. Do_execve was called in Sys_execve.

//sys_execveSYSCALL_DEFINE3(execve,                const char __user *, filename,                //可执行程序的名称                const char __user *const __user *, argv,      //程序的参数                const char __user *const __user *, envp)      //环境变量{    return do_execve(getname(filename), argv, envp);}//do_execveint do_execve(struct filename *filename,    const char __user *const __user *__argv,    const char __user *const __user *__envp){    struct user_arg_ptr argv = { .ptr.native = __argv };    struct user_arg_ptr envp = { .ptr.native = __envp };    return do_execve_common(filename, argv, envp);}

It is clear that you continue to parse the Do_execve_common called:

static int Do_execve_common (struct filename *filename, struct user_arg_ptr argv, struct user     _arg_ptr envp) {struct LINUX_BINPRM *bprm;                 struct file *file;    struct Files_struct *displaced;    int retval; ... bprm = kzalloc (sizeof (*BPRM), gfp_kernel);//Allocate a LINUX_BINPRM structure on the heap ... file = do_open_exec (filename);//open need to load                             Executable file, file contains information about an open executable file ... bprm->file = file;                   Assignment file pointer bprm->filename = BPRM->INTERP = filename->name;//Assignment filename retval = bprm_mm_init (BPRM);  Create the memory address space for the process ... BPRM->ARGC = count (argv, max_arg_strings);//number of assignment parameters ... BPRM->ENVC = count (envp, max_arg_strings);//The number of assigned environment variables ... retval = Copy_strings_kernel (1, &bprm->filename, BPRM);                         Get file path from kernel space; bprm->exec = bprm->p; P is the current memory page with the highest address retval = Copy_strings (BPRM->ENVC, ENVP, BPRM);//Copy environment variable to BPRM ... retval = Copy_strinGS (BPRM->ARGC, argv, BPRM);//Copy command-line parameters to BPRM ... retval = EXEC_BINPRM (BPRM);//Process executable ... return retval;} 

The LINUX_BINPRM structure is used to hold information about the file to be executed, such as the first 128 bytes of the file, the filename, command-line arguments, environment variables, file paths, memory descriptor information, and so on. The EXEC_BINPRM function saves the current PID, which ret = search_binary_handler(bprm); calls Search_binary_handler to find the appropriate handler for the executable file.

int search_binary_handler(struct linux_binprm *bprm) {    bool need_retry = IS_ENABLED(CONFIG_MODULES);    struct linux_binfmt *fmt;    int retval;    ...    read_lock(&binfmt_lock);    list_for_each_entry(fmt, &formats, lh) {            //遍历文件解析链表           if (!try_module_get(fmt->module))                   continue;           read_unlock(&binfmt_lock);           bprm->recursion_depth++;                //解析elf格式执行的位置           retval = fmt->load_binary(bprm);// 加载可执行文件的处理函数           read_lock(&binfmt_lock);           ...        }    return retval;

LINUX_BINFMT structure defines some function pointers, and different Linux acceptable target file formats (such as load_binary,load_shlib,core_dump) use different functions for loading the target files. Each of the LINUX_BINFMT structures corresponds to a binary program processing method. These struct instances are joined to the init_elf_binfmt kernel's corresponding format linked list by registering, register_binfmt() unregister_binfmt() inserting and deleting objects in the linked list.

struct linux_binfmt {    struct list_head lh;    struct module *module;    int (*load_binary)(struct linux_binprm *);//用于加载一个新的进程（通过读取可执行文件中的信息）    int (*load_shlib)(struct file *);         //用于动态加载共享库    int (*core_dump)(struct coredump_params *cprm);//在core文件中保存当前进程的上下文    unsigned long min_coredump;      };

The format of the target file is elf, so retval = fmt->load_binary(bprm); Load_binary actually calls Load_elf_binary to finish claiming, loading, and starting the elf binary image. Load_elf_binary This function pointer is contained in a struct named Elf_format:

static structlinux_binfmt elf_format = {          .module             =THIS_MODULE,          .load_binary     = load_elf_binary, //函数指针          .load_shlib        = load_elf_library,          .core_dump     = elf_core_dump,          .min_coredump        = ELF_EXEC_PAGESIZE,  

The global variable Elf_format assigns a pointer to register the variable in the INIT_ELF_BINFMT to the file parse list, and the corresponding file format can be found in the list. Continue analyzing Load_elf_binary:

 static int load_elf_binary(struct linux_binprm *bprm){    ...    if (elf_interpreter) {                        // 动态链接的处理           ...          } else {                                 // 静态链接的处理                    elf_entry =loc->elf_ex.e_entry;                   ...                  }           }      ...    //将ELF文件映射到进程空间中，execve系统调用返回用户态后进程就拥有了新的代码段、数据段。    current->mm->end_code = end_code;      current->mm->start_code =start_code;      current->mm->start_data =start_data;      current->mm->end_data = end_data;      current->mm->start_stack =bprm->p;      ...    start_thread(regs, elf_entry, bprm->p);}

The entry point Address field in the Elf file indicates the entry address of the program, which is typically 0x8048000 (0x8048000 above the kernel segment memory). The entry address is parsed and stored in Elf_ex.e_entry, so the starting position of the static linker is elf_entry. There is also a key point in this function Start_thread:

start_thread(struct pt_regs *regs, unsigned long new_ip, unsigned long new_sp){    set_user_gs(regs, 0);    regs->fs        = 0;    regs->ds        = __USER_DS;    regs->es        = __USER_DS;    regs->ss        = __USER_DS;    regs->cs        = __USER_CS;    regs->ip        = new_ip;               regs->sp        = new_sp;              regs->flags     = X86_EFLAGS_IF;    set_thread_flag(TIF_NOTIFY_RESUME);}

Regs the part of the Save_all macro that is pressed into the kernel stack for system calls. The value of the new_ip is equal to the value of the parameter elf_entry, that is, the start address of the main function defined in the Elf file is assigned to the EIP register, and the execution position of the process returned to the user state is changed from the next instruction of the original int 0x80 to the new_ip position.

To summarize, the call order is EXEC_BINPRM, Do_execve_common-Do_execve, Sys_execve-I, when the system call returns from the kernel state to the user state, the EIP jumps directly to the ELF program's entry address, The CPU also gets a new user-state stack (containing command-line arguments for the new program and Shell context). In this way, the new program begins to execute.

4. Debugging of static link executable files

Cover test.c with test_exe.c, add a sentence menuconfig () to execute a program.

int Exec(int argc, char *argv[]){        int pid;        /* fork another process */        pid = fork();        if (pid < 0)        {                /* error occurred */                fprintf(stderr,"Fork Failed!");                exit(-1);        }        else if (pid == 0)        {                /*       child process  */        printf("This is Child Process!\n");                execlp("/hello","hello",NULL);        }        else        {                /*      parent process   */        printf("This is Parent Process!\n");                /* parent will wait for the child to complete*/                wait(NULL);                printf("Child Complete!\n");        }}

Makefile made some modifications, compiled the HELLO.C, and put both Init and hello into the rootfs.img when generating the root file system. This will automatically load the hello executable when EXECVE is executed:

Set a breakpoint at the key point of the previous analysis, and follow the execution process with a sentence down the side. Tracked start_thread , used po new_ip , and got 0x804887f.

readelf –h helloThis executable program can be seen with its entry point address also 0x804887f.

5. Problems encountered and solutions

(1) After watching the video on the dynamic link of the second way still understand fuzzy, through the search data to solve.

?? If you are calling to load a shared library dynamically, use the Dlopen defined in Dlfcn.h. Given the filename libdllibexample.so and the flag Rtld_now open the dynamic-link library and return the handle handle. The Dlsym function is used in conjunction with the Dlopen function above, which returns the address of the symbol, depending on the operation handle (the pointer returned after the dynamic link is opened by Dlopen) Dynamicloadinglibapi the symbol (the name of the function or global variable that is required to get it) and the handle. Use this address to get the address of a specific function in the library and call the corresponding function in the library. This allows you to use dynamically loaded functions defined within the shared library.

(2) Do not understand the PO in the commissioning

The PO is an abbreviation for print_object, which can output not only the object that displays the definition, but also the result of the expression. I tried P, Po, p/d, p\x, and compared the results of their execution:

You can see that the P, p/d (10 binary), p\x (16) Output values are preceded by a prefix like "$1=", which are variables that can be used in subsequent expressions, and the PO cannot store its return value in a variable. As for the PO can also be seen in which places are not very clear, later encountered a detailed analysis.

(3) in the sixth week experiment, the rename function realizes "HELLO.C" is renamed "NEWHELLO.C", in the current folder to put a hello.c file can be achieved. But on Menuos, the hello.c file is attempted to be placed under the menu folder, the execution Rename command display is unsuccessful:

So I never knew where to put the hello.c file before I could rename it successfully. This Zhou Meng teacher modified Makefile file reminds me, I modified the makefile, the HELLO.C package into the image file:

Although the display performed successfully, unfortunately HELLO.C was not renamed to NEWHELLO.C:

I think, the change should be rootfs.img in the hello.c, so here hello.c not be modified (do not know the right thinking, want to open rootfs.img, tried several methods are not resolved).

Second, the textbook notebook virtual file system

1. The virtual file system (VFS) is the abstraction layer between the Linux kernel and the storage device. There are four main object types in the VFS, namely the Super Block object, the index node object, the Catalog item object, and the file object.

2. The Super block primarily stores information about specific file systems, which are stored on disk and created in memory when used. For disk file systems, this object typically corresponds to a file system control block (disk Super block) on the disk.

3. The index node contains all the information the kernel needs to manipulate the file or directory. An index node represents a file in the filesystem (the file here refers not only to ordinary files that we normally think of, but also to directories, special device files, and so on). The index node is stored on disk and is created in memory when it is accessed by the application.

4. Through the index node has been able to locate the specified file, but the index node object is very many properties, in the search, compare files directly with the index node efficiency is not high, so introduced the concept of directory entry (dentry). The catalog item does not actually exist on disk, and when used, creates a catalog item object in memory.

5. In a file path, each part of the path is called a catalog item. Each catalog item object has 3 states that are used, unused, and negative states. A used directory entry corresponds to a valid index node, and the object is made up of one or more users; an unused directory entry corresponds to a valid index node, but VFS does not currently use this directory entry; A negative state directory entry does not have a valid index node.

6. In Linux, in addition to ordinary files, other items such as directories, devices, sockets and so on are also treated as files, "all documents". The file object represents a file that the process has opened, and from the user's point of view, we are manipulating a file object in the code. Although the file object corresponding to a file is not unique, its corresponding index node and directory item object are unique.

The 7.VFS also has 2 file system-specific 2 objects that struct file_system_type describe a variety of particular file system types, such as EXT3,EXT4 or UDFs, struct vfsmount to describe an instance of an installation file system. A file system supported by Linux has and has only one file_system_type structure, regardless of whether it has 0 or more instances installed in the system. When the file system is actually installed, a VFSMOUNT structure is created at the installation point.

8. The following 3 structures and processes are closely linked:

• struct files_struct: Refers to the files directory entry in the process descriptor, where all information related to a single process, such as open files and file descriptors, is included.
• struct fs_struct: Refers to the FS domain in the process descriptor, which contains information about the file system and process.
• struct Mmt_namespace: Is pointed to by the Mmt_namespace field in the process descriptor.

Block I/O layer

The 1.I/O device mainly has the character device and the block device, the block device can read and write the contents of the device randomly, compared to the character device's content in sequential reading and writing device. Character devices can only be accessed sequentially, and block devices are randomly accessed.

2. The smallest addressable unit of a block device is a sector. The size of the sector is generally 2 integer times, and the most common size is 512 bytes. A sector is the basic unit of all block devices, which cannot be addressed and manipulated by the block device compared to its smaller units. Although physical disk addressing is done at the sector level, all disk operations performed by the kernel are done in blocks. To facilitate file system management, the block size is generally an integer multiple of the sector, and is less than or equal to the size of the page.

3. When a block is transferred into memory, it is stored in a buffer. Each buffer corresponds to a block, which is equivalent to the representation of a disk block in memory. Each buffer has a corresponding descriptor, represented by the buffer_head struct, called the buffer header, which contains all the information required for internal and operational buffers.

The 4.bio struct represents all the memory pages involved in an I/O operation. The bio struct also guarantees the execution of I/O operations to the kernel by describing the buffer with fragments, even if a buffer is scattered across multiple locations of memory.

5.bio corresponds to a page in memory, and the buffer header corresponds to a block in the disk.

6. Block devices store their pending block I/O requests in the request queue, which is represented by the request_queue struct. Each item in the Request queue table is a separate request, represented by the reques struct. Because a request may operate on multiple contiguous disk blocks, all requests can be made up of multiple bio structures.

7. Although the blocks on the disk must be contiguous, these blocks in memory do not necessarily have to be contiguous.

The job of the 8.I/O Scheduler is to manage the request queue for block devices. Reduce disk addressing time by merging and sorting.

9. In order to ensure the efficiency of disk addressing, it is generally possible for the head to move in one direction, so that the head and then back to move, so as to shorten the total disk addressing time of all requests, I/O scheduler called elevator scheduling.

There are 4 I/O schedules built into the 10.2.6 kernel: predictive (AS), fully-justified queueing (CFQ), Deadline (deadline), and empty operations (NOOP). Use the command-line option elevator=xxx to enable either of these.

2017-2018-1 20179202 "Linux kernel Fundamentals and analysis" Eighth week assignment