Information Security System Design Foundation 13th Week Study Summary

12th Chapter Concurrency Process

Applications that use application-level concurrency are called concurrent programs.

Three basic ways to construct concurrent programs: process, I/O multiplexing, threading.

12.1 Process-based concurrency programming

1. The server accepts connection requests from clients

2. The server derives a child process for this client service

3. The server accepts another connection request

4. Server derivation another child process for new client service

12.1.1 process-based concurrent servers

Attention:

1. Because the server will typically run for a long time, include a sigchild handler to reclaim the zombie child process resources.

2. Parent-child processes must close their respective CONNFD copies, and the parents process must close its connected descriptor to avoid memory leaks.

3. The connfd of the parent-child process is closed and the connection to the client terminates.

12.1.2 about the pros and cons of the process

The process shares a file table and does not share the user address space so that a process cannot accidentally overwrite the virtual memory of another process, but a separate address space makes it more difficult for a process to share state information.

12.2 Concurrency process based on I/O multiplexing

I/O multiplexing technology: Use the Select function to require the kernel to suspend a process and return control to the application only after one or more I/O events have occurred.

The Select function handles a collection of type Fd_set, also known as a descriptor collection. Logically, we consider a descriptor set as a bit vector of size n.
Each bit BK corresponds to the descriptor K. When and only if bk= 1, the descriptor K indicates that it is an element of the descriptor collection. Only allow you to do three things on the descriptor set: 1) assign them, 2) Assign one of these types of variables to another variable, 3) modify and examine them with Fd_zero, Fd_set, FD_CLR, and Fd_isset macros.

The SELECT function has two losers: a descriptor set called a read collection (Fdset) and the cardinality (n) of the Read collection (which is actually the maximum cardinality of any descriptor collection). The Select function blocks until at least one descriptor in the Read collection is ready to be read. When and only if a request to read a byte from the descriptor is not blocked, the descriptor K indicates that it is ready to be read. As a side effect, select modifies the fd_set that the parameter Fdset points to, indicating a subset of the Read collection called Ready set, which is made up of a read-ready descriptor in the reading set. The value returned by the function indicates the cardinality of the prepared collection. Due to this side effect, we must update the Read collection every time we call SELECT.

12.2.1 concurrency Event-driven server based on I/O multiplexing

I/O multiplexing can be used as the basis for concurrent event drivers, in which the flow progresses because of some sort of event. The general concept is to model the logical flow as a state machine.

A state machine is a set of states, input events, and transfer him, where the transition is to map state and input events to states. Each transfer maps one (input state, input event) pair to an output state. Self-loops are transitions between the same input and output states. The state is turned into a graph with the nodes representing the state, the arcs representing the transitions, and the markings on the arcs representing the input events. A state machine executes from a certain initial state. Each input event raises a transition from the current state to the next state.

The server uses I/O multiplexing and detects the occurrence of input events with the Select function.

The server calls the Select function to detect two different types of sender events: a) A connection request from a new client arrives, B) a client's existing connection descriptor is ready to read.

When a connection request arrives, the server opens the connection, calls the Add_client function, adds the user to the pool, and finally, the server calls the Check_clients function, returning a line of text from each prepared connection descriptor.

Advantages and disadvantages of 12.2.2 I/O multiplexing technology

Advantages:

1, gives the programmer more control over the program behavior.

2. Each logical stream can access the entire address space of the process, and sharing data becomes easy.

Disadvantages:

Complex coding

12.3 Thread-based concurrency programming

Thread: A logical flow that runs in the context of a process.

12.3.1 Threading Execution Model

Each process begins its life cycle as a single thread, which is called the main thread. At some point, the main thread creates a peer thread that, starting at this point in time, runs concurrently on two threads. Finally, because the main thread executes a slow system call. Or because it is interrupted by the system's interval timer, the control is passed through the context switch to the peer thread. The peer thread executes for a period of time, then the control is passed back to the main thread, and so on.

Because the context of a thread is much smaller than the context of a process, the context switch of a thread is much faster than the context switch of a process. The other difference is that threads are not organized as a process, not by a strict parent-child hierarchy. A thread that is associated with a process makes up a peer (thread) pool, independent of the threads created by other threads.

The main thread differs from other threads only in that it is always the first one running in the process. The primary impact of the concept of peer-pooling is that a thread can kill any of its peer threads, or wait for any of its peers to terminate. In addition, each peer thread can read and write the same shared data.

12.3.2 Posix Threads

A Posix thread is a standard interface for processing threads in a C program. Pthreads defines approximately 60 functions that allow the program to create, kill, and reclaim threads, share data securely with peer threads, and also notify peer-to system state changes.

The code of the thread and the local data are encapsulated in a threading routine (thread routine). If you want to pass multiple arguments to the money-path routine, you should place the parameters in a structure and pass a pointer to that structure. If you want the thread routines to return multiple parameters, you can return a pointer to a struct.

12.3.3 Creating Threads

The Pthread_create function creates a new thread and takes an input variable, ARG, to run the thread example F in the context of the new thread. You can use the attr parameter to change the default properties of the newly created thread.

When Pthread_create returns, the parameter TID contains the ID of the newly created thread. A new thread can get its own thread ID by calling the Pthread_self function.

12.3.4 terminating a thread

A thread is terminated in one of the following ways:

1. When the thread routine of the when top layer returns, the line Cheng terminates.

2, by calling the Pthread_exit function, the line Cheng terminated. If the main thread calls Pthread_exit, it waits for all other peer threads to terminate before terminating the main thread and the entire process, and the return value is Thread_return.

3. A peer thread calls the Unix Exit function, which terminates the process and all threads associated with the process.

4. Another peer thread terminates the current thread by calling the Pthread_cancle function with the current thread ID as a parameter.

12.3.5 recovering resources for terminated threads

The thread waits for another thread to terminate by calling the Pthread_join function.

The Pthread_join function blocks until the thread TID terminates, assigns the (void*) pointer returned by the thread routine to the location that the Thread_return points to, and then reclaims any memory resources that the terminated thread occupies.

The Pthread join function can only wait for a specified thread to terminate.

12.3.6 Separating threads

At any point in time, threads are either associative or detached. A binding thread can be recalled by other threads and killed by its resources. Its memory resources (such as stacks) are not released until it is reclaimed by another thread. Instead, a disconnected thread cannot be recycled or killed by other threads. Its memory resource is automatically released by the system when it terminates.

By default, threads are created to be combined. To avoid memory leaks, each binding thread should either be Cheng by another thread or be separated by calling the Pthread_detach function.

Pthread_detach function separation can be combined with thread tid. Threads are able to detach themselves by using Pthread_detach pthread_self () as parameters.

12.3.7 Initializing Threads

The Pthread_once function allows you to initialize the state associated with a thread routine.

The Once_control variable is a global or static variable that is always initialized to Pthread_once_init. When you first call Pthread_once with parameter Once_control, it calls Init_routine, which is a function that has no input parameters or returns anything.

The Pthread_once function is useful when you dynamically initialize a global variable that is shared by multiple threads.

12.3.81 thread-based concurrent servers

How to pass the connected descriptor to the peer thread when pthread_ create is called. The most obvious way is to pass a pointer to this descriptor.

The peer thread indirectly refers to the pointer and assigns it to a local variable.

This can be an error because it introduces a competition between the peer-to assignment statement and the Accept statement of the main thread. If the assignment statement is completed before the next accept, the local variable CONNFD in the peer thread gets the correct descriptor value. However, if the assignment statement is completed after the accept, the local variable CONNFD in the peer thread gets the next concatenated descriptor value. The unfortunate result is that two threads now perform input and output on the same descriptor.

To avoid this potentially deadly competition, we must allocate the connected descriptor returned by each accept to its own dynamically allocated memory block to avoid memory leaks in thread routines. If you do not explicitly retract a thread, you must detach each thread so that its memory resources can be retracted when it terminates. Further, we must carefully release the memory block allocated by the main thread.

12.4 Shared variables in multi-threaded programs

12.4.1 Thread Memory model

A set of concurrent threads runs in the context of a process. Each thread has its own separate thread context, including thread ID, stack, stack pointer, program counter, condition code, and general purpose register value. Each thread shares the remainder of the process context with other threads. This includes the entire user virtual address space, which consists of read-only text (code), read/write data, heaps, and all shared library code and data regions. Threads also share the same set of open files.
It is not possible to have one thread read or write the register value of another thread. On the other hand, any thread can access any location of the shared virtual storage.

is stored in the stack area of the virtual address space and is usually accessed independently by the corresponding thread.

12.4.2 Mapping variables to memory

In threaded C programs, variables are mapped to virtual storage according to their storage type:

Global variables.

Local automatic variables.

Local static variable.

12.4.3 Shared variables

A variable v is a co-kiosk when and only if one of its instances is referenced by more than one thread.

12.5 Synchronizing Threads with semaphores

Break the loop code of thread I into five parts:

Hi: instruction block in the loop head

Li: Loads the shared variable cnt-to-register%eaxi instruction, where%eaxi represents the value of the register%eax in thread i.

Ui: Update (ADD)%eaxi instructions.

Si: Store The updated value of%eaxi back to the instruction of the shared variable CNT.

Ti: the instruction block at the tail of the loop.

Note that the head and tail only manipulate local stack variables, while the Li, Ui, and SI operations share the contents of the counter variable.

In general, you have no way to predict whether the operating system will choose the correct order for your thread.

A Progress graph (progress Graph) method to illustrate these concepts of correct and incorrect order of instruction.

12.5.1 Progress Chart

A progress graph models the execution of n concurrent threads into a trajectory line in an n-dimensional Cartesian space. Each axis k corresponds to the progress of thread K. Each delegate thread K has completed the state of instruction Ik. The origin of the graph corresponds to the initial state of no thread completing an instruction.

A progress map models the instruction execution into a transition from one state to another. A transform is represented as a forward edge from one point to an adjacent point. The valid conversion is to the right (one instruction in thread 1 completes) or up (one instruction in thread 2 completes). Two instructions cannot complete the one by one diagonal conversion at the same time is not allowed. The program will never run in reverse, so a shift down or to the left is illegal.

The execution history of a program is modeled as a trace line in the state space.

The trajectory line around the unsafe area is called the safety trajectory line. In contrast, a trace line that touches any unsafe area is called an unsafe trajectory.

12.5.2 Signal Volume

A method for synchronizing different threads of execution, based on a special type of variable called a semaphore. The semaphore S is a global variable with nonnegative integer values, which can only be handled by two special operations, called P and V:

P (s): if S is nonzero, p will subtract s by 1 and return immediately. If S is zero, then the thread is suspended until s becomes nonzero, and a V operation restarts the thread. After a restart, the P operation will subtract s by 1 and return control to the caller.

V (s): v operation will add S 1. If any of the threads are blocked while the P operation waits for s to become nonzero, then the V operation restarts one of these threads, and the thread will then subtract s by 1 to complete its P operation.

12.5.3 using semaphores to achieve mutual exclusion

The basic idea is to associate each shared variable (or a set of related shared variables) with a semaphore s (initial 1) and then surround the corresponding critical section with P (s) and V (s) operations.

The semaphore that protects the shared variable is called the two-dollar semaphore (binary semaphore) because its value is always 0 or 1. A two-dollar semaphore that is intended to provide mutual exclusion is often also referred to as a mutex (mutex). Performing P operations on a mutex is known as locking the mutex. Similarly, performing a V operation is known as unlocking the mutex lock. A thread that has a lock on a mutex but has not yet been unlocked is called an exclusive lock. A semaphore that is used as a counter for a set of available resources is called a count semaphore.

The key idea is that this combination of P and V operations creates a set of states called the Forbidden Zone (forbidden region), where S<o. Because of the invariance of the semaphore, there is no practical trajectory line that can contain the state in the forbidden area. Moreover, since the Forbidden Zone completely includes the unsafe zone, there is no practical track line that can touch any part of the unsafe zone. As a result, each practical trajectory is safe, and the program correctly increments the value of the counter regardless of the order of the runtime directives.

Use the semaphore to mutually exclusive. The infeasible state of the S<o defines a forbidden zone, which completely includes the unsafe zone, preventing the actual feasible trajectory line from contacting the unsafe zone.

The Forbidden Zone created by the P and V operations makes it impossible to have multiple threads executing instructions at any point in time, in the bounding critical section. In other words, the semaphore operation ensures mutually exclusive access to the critical section.

12.5.4 using semaphores to dispatch shared resources

In addition to providing mutexes, another important function of semaphores is to dispatch access to shared resources.

1. Producer-Consumer issues

Producer one consumer problem. Producers generate projects and insert them into a limited buffer. Consumers take these items out of the buffer and then consume them.

Because inserting and fetching items involves updating shared variables, we must ensure that access to the buffers is mutually exclusive. But it is not enough to guarantee exclusive access, we also need to schedule access to the buffer. If the buffer is full (no empty slots), then the producer must wait until a slot becomes available. Similarly, if the buffer is empty (no usable items), then the consumer must wait until a project becomes available.

Construct a producer-consumer program using SBUF: A concurrent server based on pre-threading.

The Sbup operation type is a finite buffer of sbuf_t, and the item is stored in a dynamically allocated N-item integer array (BUF). The front and rear index values record the first and last items in the array. Three semaphores synchronize access to the buffer. The mutex semaphore provides mutually exclusive buffer access. The slots and items semaphores record the number of empty slots and available items, respectively.

The Sbuf_init function allocates heap memory for buffers, sets front and rear to represent an empty buffer, and assigns an initial value to three semaphores. This function is called once before calling any of the other three functions.

The Sbuf_deinit function is the buffer that is stored when the application finishes using the buffer. The Sbuf_insert function waits for an available slot, locks the mutex, adds items, unlocks the mutex, and then announces that a new item is available.

The Sbuf_remove function is symmetric with the Sbuf_insert function. After waiting for an available buffer item, lock the mutex, remove the item from the front of the buffer, unlock the mutex, and then signal a new slot to be used.

2. Reader-writer question

Starvation: A thread is blocked indefinitely and cannot progress.

12.5.5 Synthesis: Pre-threading-based concurrent servers

The organizational structure of a pre-threaded concurrent server. A set of existing threads constantly fetching and processing a connected descriptor from a finite buffer:

12.7 Other concurrency issues

12.7.1 Thread Safety

A thread is secure, and it always produces the correct result when it is called repeatedly by multiple concurrent threads.

Four disjoint thread unsafe function classes and Countermeasures:

Functions that do not protect shared variables--protect shared variables with synchronous operations such as P and V

A function that maintains a state that spans multiple calls-calls Rand repeatedly from a single thread.

A function that returns a pointer to a static variable--① overridden; ② uses the lock-copy technique.

Functions that call thread unsafe functions

12.7.2 Re-entry

When they are called by multiple threads, no shared data is referenced.

Explicitly reentrant: All function arguments are pass-through, no pointers, and all data references are local automatic stack variables, without reference to static or full-play variables.

Implicitly reentrant: The calling thread carefully passes pointers to non-shared data.

12.7.3 using existing library functions in a thread-in-line program

The reentrant version of the thread unsafe function is used, and the name ends with the suffix _r.

12.7.4 Competition

Competition occurs when the correctness of a program relies on one thread to reach the X point in its control flow before another thread reaches the y point.
The competition occurs because the programmer assumes that the thread will pass through the execution state space according to a particular track line, forgetting another guideline that the threaded program must work correctly for any viable trace line.

To eliminate the competition, you can dynamically assign a separate block to each integer ID and pass it to the thread routine a pointer to the block, noting that the thread routines must release the blocks to avoid memory leaks.

12.7.5 deadlock

Semaphores lead to a potentially nasty run-time error called a deadlock (deadlock), which refers to a set of threads that are blocked, waiting for a condition that will never be true.

Programmers use P and V to operate improperly so that the forbidden areas of two semaphores overlap. If one of the execution trajectories happens to have reached the deadlock state D, then no further progress can be made, as overlapping forbidden areas block progress in each of the legal directions. In other words, the program is deadlocked because each thread waits for another thread to perform a V operation that is not possible at all.

Overlapping forbidden areas cause a set of States called dead regions. If a trajectory line happens to reach a state in a deadlock area, then deadlocks are unavoidable. Trace lines can enter the human deadlock zone, but they cannot be left.

Deadlock is a rather difficult problem because it is not always predictable. Some lucky lines of execution will bypass the deadlock area, while others will fall into the area.

There are many reasons for deadlock in the program, and it is generally difficult to avoid deadlocks. However, when a two-dollar semaphore is used to achieve mutual exclusion, the following simple and effective rules can be applied to avoid deadlocks:

Mutex lock Order rule: If for each pair of mutex (s, T) in the program, each thread that occupies both S and T is locked in the same order, then the program is deadlock-free.

The 11th Chapter Network programming

One, client-server model

1. When a client needs a service, it sends a request to the server to initiate a transaction.

2. After the server receives the request, it interprets it and operates its resources in an appropriate manner.

3. The server sends a response to the client and waits for the next request.

4. The client receives a response and processes it

Second, the network

Clients and servers typically run on different hosts and communicate through the computer's hardware and software resources.

With some cables and small boxes called bridges, multiple Ethernet segments can be connected to a larger LAN, known as bridging Ethernet.

The Network Bridge leverages the cable bandwidth more fully than the hub.

Multiple incompatible LANs can be connected by a special computer called a router to form an Internet (internetwork).

Third, the global IP Internet

Each Internet host runs software that implements the TCP/IP protocol, which is supported by almost every modern computer system.

The Internet's client and server mix uses socket interface functions and Unix I/O functions to communicate. Socket functions are typically implemented as system calls that fall into the kernel and invoke TCP/IP functions of various kernel modes.

1. IP Address

An IP address is a 32-bit unsigned integer.

The HTONL function converts a 32-bit integer from host byte order to network byte order.

The Ntohl function converts a 32-bit integer from the network to the host byte.

The htons and NTOHS functions perform the corresponding conversions for 16-bit integers.

IP addresses are usually represented by a decimal notation called dotted notation.

"N" represents the network. "A" denotes the application (application). and "to" represents the conversion.

2. Internet domain name

An IP address is used when the Internet client and server communicate with each other.

A subtree is called a subdomain. The first layer in the hierarchy is an unnamed root node. The next layer is a group of first-level domain names.

Common first-tier domain names include com, edu, gov, org, and net.

The next layer is a level two domain name, such as CMU. Edu

Once an organization has a two-level domain name, it can create any new domain name in the subdomain.

The internet defines the mappings between the domain name collection and the set of IP addresses.

Internet applications retrieve arbitrary host entries from the DNS database by invoking the gethostbyname and GETHOSTBYADDR functions.

In the simplest case, there is a one by one mapping between a domain name and an IP address.

In some cases, multiple domain names can be mapped to the same IP address

Most often, multiple domain names can be mapped to multiple IP addresses

Some legitimate domain names are not mapped to any IP address

3. Internet connection

Internet clients and servers communicate by sending and receiving byte streams on the connection.
The connection is point-to-point.
It is full-duplex and reliable from the point of view where the data can be both bidirectional.

The WEB server typically uses port 80, and the e-mail server uses port 25.

Iv. Socket Interface

A socket interface is a set of functions that are combined with Unix I/O functions to create a network application.

1.socket function

The client and server use the socket function to create a socket descriptor.

2.connect function

The client establishes a connection to the server by calling the Connect function.

3.OPEN_CLIENTFD function

The OPEN_CLIENTFD function establishes a connection to the server running on host hostname and listens for connection requests on well-known port ports. It returns an open socket-to-be descriptor, which is ready for input and output with Unix I/O functions.

4.bind function

Bind, listen, and accept are used by the server to establish a connection with the client.

The BIND function tells the kernel to associate the server socket address and socket descriptor SOCKFD in MY_ADDR. The parameter addrlen is sizeof (sockaddr_in).

5.listen function

The client is the active entity that initiates the connection request. The server is the passive entity that waits for a connection request from the client. By default, the kernel considers the descriptor created by the socket function to correspond to the active socket, which exists on a connected client.

The Listen function converts SOCKFD from an active socket to a listening socket (listening socket) that accepts connection requests from the client.

6.OPEN_LISTENFD function

The socket, bind, and listen functions are combined into one called. Auxiliary functions for OPEN_LISTENFD.

7.accept function

The Accept function waits for a connection request from the client.

The Accept function waits for the connection request from the client to reach the listener descriptor LISTENFD, and then fills in addr with the client's socket address and returns a connection descriptor that used by itself is used to communicate with the client using the Unix I/O function.

A listener descriptor is an endpoint that is a client connection request.

V. Web server

1. Web Foundation

The interaction between the WEB client and the server is a text-based application-level protocol called HTTP (hypertext Transfer Protocol, Hypertext Transfer Protocol). HTTP is a simple protocol. A WEB client (that is, a browser) opens an Internet connection to the server and requests some content. The server responds to the requested content, and then closes the connection. The browser reads the content and displays it on the screen.

2. Web Content

Each content returned by the WEB server is associated with a file that it manages. Each of these files has a unique name, called a URL.

3. HTTP Transactions

Because HTTP is based on lines of text that are transmitted over an Internet connection, we can use the Unix telnet program to perform transactions with any WEB server on the Internet.

An HTTP request consists of a request line, followed by 0 or more requests headers (request header) followed by an empty line of text to terminate the list of headers

HTTP supports many different methods, including GET, POST, OPTIONS, HEAD, PUT, DELETE, and TRACE.

An HTTP response consists of a response line followed by 0 or more response Headers (response header) followed by a blank line of the terminating header followed by a response body (response body). Response

The status code is a three-bit positive integer that indicates the processing of the request. The status message gives an English description equivalent to the error code.

Resources

Textbooks

Information Security System Design Foundation 13th Week Study Summary