Python3 full stack development-complementary UDP socket, operating system, concurrency theory base

One, UDP-based sockets

A simple example of UDP sockets

Import socketip_port= ('1.1.1.1', 8181) BUFSIZE=1024udp_server_client =Socket.socket (socket.af_inet,socket. SOCK_DGRAM) Udp_server_client.bind (ip_port) while True:    msg,addr=udp_server_ Client.recvfrom (BUFSIZE) print(msg,addr)    udp_server_client.sendto (Msg.upper (), Addr)

UDP service Side

ImportSocketip_port=('1.1.1.1', 9000) BUFSIZE=1024udp_server_client=Socket.socket (Socket.af_inet,socket. SOCK_DGRAM) whiletrue:msg=input ('>>:'). Strip ()if  notMsgContinueudp_server_client.sendto (Msg.encode ('Utf-8'), Ip_port) back_msg,addr=Udp_server_client.recvfrom (BUFSIZE)Print(Back_msg.decode ('Utf-8'), addr)

UDP Client

Time server

 fromSocketImport* fromTimeImportStrftimeip_port=('127.0.0.1', 9000) BufSize=1024Tcp_server=socket (Af_inet,sock_dgram) tcp_server.bind (ip_port) whiletrue:msg,addr=Tcp_server.recvfrom (bufsize)Print('===>', msg)if  notmsg:time_fmt='%y-%m-%d%x' Else: Time_fmt=msg.decode ('Utf-8') back_msg=strftime (TIME_FMT) tcp_server.sendto (Back_msg.encode ('Utf-8'), addr) tcp_server.close ()

NTP service side

 fromSocketImport*Ip_port=('127.0.0.1', 9000) BufSize=1024tcp_client=socket (Af_inet,sock_dgram) whiletrue:msg=input ('Please enter a time format (example%y%m%d) >>:'). Strip () Tcp_client.sendto (Msg.encode ('Utf-8'), ip_port) data=tcp_client.recv (bufsize)Print(Data.decode ('Utf-8') ) Tcp_client.close ()

NTP Client

Let's make a program that executes commands remotely based on UDP

 fromSocketImport*ImportSubprocessip_port=('127.0.0.1', 9003) BufSize=1024Udp_server=socket (Af_inet,sock_dgram) udp_server.bind (ip_port) whileTrue:#Receive MessageCmd,addr=Udp_server.recvfrom (bufsize)Print('User Commands----->', cmd)#Logical ProcessingRes=subprocess. Popen (Cmd.decode ('Utf-8'), shell=true,stderr=subprocess. Pipe,stdin=subprocess. pipe,stdout=subprocess. PIPE) stderr=res.stderr.read () stdout=Res.stdout.read ()#Send Messageudp_server.sendto (stderr,addr) udp_server.sendto (STDOUT,ADDR) udp_server.close ()

UDP non-stick packet server

 fromSocketImport*Ip_port=('127.0.0.1', 9003) BufSize=1024udp_client=socket (Af_inet,sock_dgram) whiletrue:msg=input ('>>:'). Strip () Udp_client.sendto (Msg.encode ('Utf-8'), Ip_port) data,addr=Udp_client.recvfrom (bufsize)Print(Data.decode ('Utf-8'), end="')

UDP non-stick packet client

The above program is a UDP-based socket that never occurs at run time

Ii. what is a process

As the name implies, a process is a process that is being executed. A process is an abstraction of a running program.

The concept of process originates from the operating system, is the core concept of the operating system, and is one of the oldest and most important abstract concepts provided by the operating system. Everything else in the operating system is expanded around the concept of processes.

So to really understand the process, you must know the operating system beforehand

third, the operating system1, why should have the operating system

Modern computer system is mainly composed of one or more processors, main memory, hard disk, keyboard, mouse, monitor, printer, network interface and other input.

Generally speaking, modern computer systems are a complex system.

One: If every application member has to master all the details of the system, it is impossible to write the code again (seriously affecting the programmer's development efficiency: it may take 10,000 years to master these details ...). ）

Second: and managing these components and optimizing their use is a challenging task, so computing installs a layer of software (System software), called the operating system. Its mission is to provide a better, simpler, clearer computer model for the user program and to manage all the devices just mentioned.

Summarize:

Programmers are unable to understand all the hardware operation details, the management of these hardware and optimize the use of the work is very tedious, this tedious work is the operating system to do, with him, the programmer from these tedious work out, only need to consider their own application software to write, The application software uses the functionality provided by the operating system directly to use the hardware indirectly.

2, what is the operating system

To be concise, the operating system is a control program that coordinates, manages, and controls the hardware and software resources of the computer. Where the operating system is located 1

#操作系统位于计算机硬件与应用软件之间, Nature is also a software.

The operating system consists of the kernel of the operating system (running in the kernel state, managing the hardware resources) and the system calls (running in the user state, providing the system calling interface for applications written by the application programmer),

So, simply saying that the operating system is running in the kernel state, is not accurate.

Figure 1

In detail, the operating system should be divided into two parts:

 #一: Hides the ugly hardware calling interface, providing the application programmer with a better, simpler, clearer model for invoking hardware resources (System call interface). 
 After the application has these interfaces, it is no longer necessary to consider the details of the operating hardware and concentrate on developing their own applications. For example, the operating system provides the abstract concept of files, the operation of the file is the operation of the disk, with the file we do not have to consider the disk read and write control (such as the control of disk rotation, moving the head read and write data and other details), # Second: The application to the hardware resources of the race request to become orderly for example: many applications are actually sharing a set of computer hardware, for instance, there may be three applications need to request a printer to output the content, 
 then a program competes to the printer resources to print, and then may be B competition to the printer resources, It can also be C, which causes the disorder, the printer may print a piece of content and then print C ..., one of the functions of the operating system is to order this disorder. 

# role One: Provides an abstraction of how the application uses hardware resources For example, the operating system provides the abstract concept of files, the operation of the file is the operation of the disk, with the file we do not have to consider the disk read and write control note: The operating system provides the application of the abstraction is simple, clear, elegant. Why should we provide this abstraction? Hardware manufacturers need to provide their own hardware drivers for the operating system (device drivers, which is why we have to use the sound card, you must install the sound card driver ...) ), in order to save cost or to be compatible with old hardware, their drivers are complex and ugly operating system is to hide these ugly information, so as to provide users with a better interface so that users use SHELL,GNOME,KDE see a different interface, But in fact all use the same set of Linux system provided by the abstract interface # role Two: the management of hardware resources modern operating system running multi-channel program, the task of the operating system in the competition between the process of orderly control of the processor, Memory and other i/O interface devices. For example: Run three programs on the same computer at the same time, and they three want to output the results on the same computer at the same time, then the first few lines may be the output of program 1, then a few lines are the output of program 2, and then the output of program 3. It's going to be a mess. The operating system sends the results of the printer to the buffer of the disk, and the files on the temporary disk are sent to the printer output after a program has completely finished. At the same time, other programs can continue to produce more output (the output of these programs is not actually sent to the printer), so that the operating system will be organized by the competition generated by the order of the disorder. 

detailed

Figure 2

3, the difference between the operating system and the common software

1. The main difference is: You don't want to use the storm. You can choose to use the Thunder player or simply write one yourself, but you cannot write a program that is part of the operating system (clock interrupt handler), the operating system is hardware protection, can not be modified by the user.

2. The difference between the operating system and the user program does not lie in the position of the two. In particular, the operating system is a large, complex, long-lived software,

• Large: The source code for Linux or Windows is 5 million lines in order of magnitude. According to a total of 1000 lines of 50 lines per page of the book, 5 million lines to have 100 volumes, to use an entire book shelf to set up, this is only the kernel part. User programs, such as GUIs, libraries, and basic applications such as Windows Explorer, can easily reach this number 10 times or 20 times times more.
• Longevity: The operating system is very difficult to write, such a large amount of code, once completed, the operating system owners will not easily throw away, and then write one. But on the basis of the original improvement. (basically you can see Windows95/98/me an operating system, while Windows Nt/2000/xp/vista is a two-bit operating system, which is very similar to users.) UNIX and its variants and clones have evolved over the years, such as System V, Solaris, and FreeBSD, all of which are original versions of UNIX, but although Linux is highly compatible with UNIX patterns, But Linux has a new code base)

4. History of operating system

First generation Computers (1940~1955): Vacuum tubes and punch cards

Background of the first generation of computers:

Before the first generation, humans were trying to replace manpower with machinery, and the first generation of computers was a sign that computers had entered the electronic age from the mechanical age, and since the Babbage failed until the Second World War, the construction of digital computers had barely progressed, and the Second World War had stimulated the explosive progress of computer research.

Professor John Atanasoff of Lowa State University and his student Clifford Berry built a digital computer that was considered the first to work. The machine uses 300 vacuum tubes. Around the same time, Konrad Zuse built a Z3 computer in Dublin Relay, a team from the Bletchley Park in England built Colossus,howard Aiken at Harvard University in 1944 to build Mark 1, William of the University of Pennsylvania Mauchley and his students J.presper Eckert built the ENIAC. These machines are binary, some use vacuum tubes, some are programmable, but are very primitive, it takes a few seconds to complete the simplest operation.

During this time, engineers in the same group designed, built, programmed, operated and maintained the same machine, all of which were programmed in purely machine language and even worse, requiring thousands of cables to connect to the plug-in board to control the basic functions of the machine. There is no programming language (no assembly), and the operating system has never heard of it. The process of using the machine is more primitive, see the following ' work process '

Characteristics:
No concept of operating system
All programming is directly manipulating the hardware.

Working process:
Programmer on the wall when the table appointment for a period of time, and then the programmer took his plug-in version to the computer room, will own the plug-in Board street computers, within a few hours he exclusive access to the entire computer resources, the back of a group of people have to wait (more than 20,000 vacuum tubes will often be burned out of the situation appears).

Then there was a punch card that could write the program on the card and read it into the machine without the plug-in board.

Advantages:

The programmer has the entire resource in the time period of the application, can debug his own program immediately (bug can be processed at once)

Disadvantages:

Wasting computer resources, only one person in a time period.
Note: Only one program is in memory at the same time and is executed by CPU calls, say 10 program execution, is serial

Second generation computer (1955~1965): Transistors and batch processing systems

Generation background of the second generation of computers:

Because the computer was very expensive at the time, it is natural to think of ways to reduce the wastage of the machine. The usual method is the batch processing system.

Characteristics:
Designers, production personnel, operators, program personnel and maintenance personnel directly have a clear division of labor, the computer is locked in a dedicated air-conditioned room, operated by professional operators, this is the ' mainframe '.

With the concept of an operating system

Programming languages: Fortran or assembly language, written on paper, then punched into a card, then the card cassette into the input room, to the operator, and then drink coffee waiting for the output interface

Working process: Illustrations

How the second generation solves the first generation of problems/shortcomings:
1. Save a bunch of people's input into a big wave of input,
2. Then the sequential calculation (this is problematic, but the second generation is not resolved)
3. Save a bunch of people's output into a large wave of output

The predecessor of the Modern operating system: (see figure)

Advantages: Batch processing, saving the machine

Disadvantages:
1. The entire process requires people to participate in the control, the tape to move (between the two villains)

2. The process of calculation is still a sequential calculation-"serial

3. The programmer originally enjoyed a period of time of the computer, now must be unified planning to a batch of jobs, waiting for the results and re-commissioning of the process will require the equivalent batch of other programs to work (which greatly affected the development efficiency of the program, unable to debug the program in time)

Third generation computer (1965~1980): IC Chip and multi-channel program design

Generation background of the third generation computer:

In the early 1960s, most computer manufacturers had two completely incompatible production lines.

One is word-oriented: large scientific computers, such as IBM 7094, are mainly used for scientific calculations and engineering calculations.

The other is character-oriented: commercial computers, such as IBM 1401, are primarily used by banks and insurers for tape archiving and printing services

Developing and maintaining a completely different product is expensive, while different users have different uses for the computer.

How to troubleshoot second-generation computer problems 1:
The card is taken to the machine room to quickly read the job from the card into the disk, so any time when a job ends, the operating system can read a job from the tape, loaded into the empty memory area to run, this technology is called
Simultaneous external device online operation: spooling, which is used for output simultaneously. When this technology is used, it is not necessary to IBM1401 the machine, and do not have to move the tape to move away (the middle two villain no longer needed)

How to troubleshoot second-generation computer problems 2:

The operating system of the third-generation computer has widely applied the key technology of the second generation computer's operating system: multi-Channel technology

CPU in the process of performing a task, if the need to operate the hard disk, the operation of the drive to send instructions, once issued, the hard disk on the mechanical arm sliding read data into memory, this period of time, the CPU needs to wait, the time may be very short, but for the CPU has been very long, Long enough to allow the CPU to do a lot of other tasks, if we let the CPU during this time to switch to do other tasks, so that the CPU is not fully utilized. This is the technical background of multi-channel technology

Multi-Channel technology:

Multi-channel technology in multi-Dow is a number of programs, multi-channel technology is to solve multiple programs to compete or share the same resource (such as the CPU) of the orderly scheduling problem, the solution is multiplexing, multiplexing is divided into time multiplexing and space reuse.

Spatial multiplexing : divides the memory into parts, each of which is put into a program, so that there is a multi-channel program in memory at the same time.

time Reuse : When a program waits for I/O, another program can use the CPU, and if the memory can hold enough jobs at the same time, the CPU utilization can be close to 100%, similar to the overall approach that we learned in elementary school mathematics. (The operating system uses multi-channel technology, you can control the process of switching, or the process to compete for CPU execution permissions.) This kind of switchover will not only occur when a process encounters Io, a process that consumes CPU time too long will switch, or the operating system takes the CPU's execution rights)

  Modern computers or networks are multi-user, multiple users not only share hardware, but also share files, databases and other information, sharing means conflict and disorder. The operating system is primarily used to  1 2 3 1/ o is blocked, the operating system allocates the CPU to the next program, and so on, until the first program is reassigned to the CPU and then runs again, due to the fast switching speed of the CPU, The feeling to the user is that these programs are running at the same time, or concurrent, or pseudo-parallel. As for how resources are used for time reuse, or who should be the next program to run, and how long a task needs to run, these are the work of the operating system.  2. Reuse of space each customer acquires a small fraction of the resources in a large resource, reducing the time it takes to queue up for resources. For example, multiple running programs go into memory at the same time, and the hardware layer provides a protection mechanism to ensure that the respective memory is split and controlled by the operating system, which is much more efficient than one program exclusive memory and one queue into memory. Other resources for space reuse also have disks, and in many systems, one disk saves files for many users at the same time. Allocating disk space and documenting who is using which disk block is a typical task for operating system resource management. The combination of these two approaches is multi-channel technology.  

detailed

The biggest problem with spatial multiplexing is that memory must be split between programs, which needs to be implemented at the hardware level and controlled by the operating system. If the memory is not split, then one program can access the memory of the other program.

The first loss is security, such as your QQ program can access the operating system's memory, which means that your QQ can get all the operating system permissions.

Second, the loss of stability, a program crashes when it is possible to recycle the memory of other programs, for example, the memory of the operating system is recycled, the operating system crashes.

The third-generation computer's operating system is still batch-processing

Many programmers miss their first-generation computers and can debug their own programs on the fly. To meet the programmer's quick response, a time-sharing OS appeared

How to troubleshoot second-generation computer problems 3:

Time-sharing operating system:
Multiple online terminals + multi-channel technology

20 clients loaded into memory at the same time, 17 thinking, 3 running, the CPU is in a multi-channel way to handle the memory of the 3 programs, because the customer submitted is generally short instructions and very little time-consuming, the index computer can provide many users with fast interactive services, All the users think they have the computer resources alone

Ctts: MIT has developed a successful, CTSS-compatible time-sharing system on a modified 7094 machine, and the third-generation computer has widely adopted the necessary protection hardware (the memory between the programs is isolated from each other), and the time-sharing system has become popular

MIT, Bell Labs and GE decided to develop a multics that could support hundreds of terminals at the same time (the designers focused on building a machine to meet the needs of all the users in the Boston area), and it was clear that Ctts was going to die.

Later, a computer scientist Ken Thompson, who attended the development of Multics, developed a simple, single-user version of Multics, which was later a UNIX system . Based on a number of other UNIX versions, the IEEE proposes a UNIX standard, POSIX (Portable Operating System interface Portable Operating system Interface), to enable the program to run on any version of UNIX.

Later, in 1987, a small clone of Unix, Minix, was introduced for use in teaching. Finnish student Linus Torvalds wrote Linux based on it

Fourth generation computer (1980~ to date): PC

PS: The ability to support (pseudo) concurrency can be guaranteed even if there is only one CPU available (this is true for earlier computers). Turning a single CPU into multiple virtual CPUs (multi-channel technology: Time multiplexing and spatial multiplexing + hardware support isolation), without process abstraction, modern computers will no longer exist.

# the role of an operating system: 1: Hide ugly Complex hardware interface, provide good abstraction    interface 2: Management, scheduling process, and the competition of more than one process to the hardware order # two multi-channel technology: 1. Background: For the single core, to achieve concurrent    PS:    Now the host is generally multicore, then each core will use multi-channel technology    has 4 CPUs, a program running in CPU1 encountered IO blocking, will wait until the end of the IO and re-Dispatch, will be dispatched to 4    any one of the CPUs, which is determined by the operating system scheduling algorithm.  2. Reuse in space: if there are multiple programs    in memory 3. Time reuse: Multiplexing a CPU time slice       emphasis: Encountering IO cut, taking up CPU time too long also cut, The core is to save the state of the process before it is cut, so that the            next time you switch back, you can continue to run based on where you last cut off.

The theoretical basis of concurrency

Python3 full stack development-complementary UDP socket, operating system, concurrency theory base