Linux shell Simulation multithreaded implementation of the task detailed

The shell itself cannot be multi-threaded, but it can simulate multithreading by starting the subprocess and putting the child processes into the background, in order to increase the efficiency of the script while not significantly increasing the load, but also the need to put the number of processes in the background at the same time limit.

The code is as follows

Copy Code

#!/bin/bash Set-x # Open Debug mode #判断是否有参数 If [$#!= 1];then echo "You have entered an incorrect parameter" Exit-1 Fi # Maximum number of processes allowed Max_thread_num=5 Tmp_fifo_file=/tmp/$$.fifo # as file name with the current process ID number run by script Mkfifo "$tmp _fifo_file" # Create a new random FIFO pipeline file exec 9<> "$tmp _fifo_file" # definition file descriptor 9 point to this FIFO pipe file RM "$tmp _fifo_file" # write the specified number of newline characters in advance to the FIFO pipeline file, a newline character represents a process For ((i=0;i< $MAX _thread_num;i++));d o Echo Done >&9 # loop read the URL and judge the status code While Read line Todo { # Process Control Read-u 9 # Reading rows from file Descriptor 9, actually pointing to FIFO pipeline { isok= ' curl-i-l-m 60-o/dev/null-s-W%{http_code} $line ' If ["$isok" = "];then" echo $line "OK" Else echo $line $isok Fi Echo >&9 #, at the end of the current process, to write a blank line to the FIFO pipeline file }& } Done < A wait echo ' execution end ' EXEC 9>&-# delete file descriptor 9 Exit 0

The task of the script is to judge the URL in a list of URLs, to determine whether the URLs can continue to be accessed, by Curl to obtain the status code of the HTTP.

The statements in the red Section {} above are put into the subprocess to execute in the background. When the FIFO 5 empty lines read, the loop continues to wait for read in the FIFO data, when the background of the child process completes the task, queued to FIFO input blank line, so that the FIFO data, cycle continue to execute.

Let's look at the results of shell execution

The code is as follows

Copy Code

# bash scanurl.sh url.txt + ' [' 1 '!= ' 1 '] ' + max_thread_num=5 + tmp_fifo_file=/tmp/111cn.net + mkfifo/tmp/111cn.net + EXEC + rm/tmp/111cn.net + ((i=0)) + ((i<5)) + Echo + ((i++)) + ((i<5)) + Echo + ((i++)) + ((i<5)) + Echo + ((i++)) + ((i<5)) + Echo + ((i++)) + ((i<5)) + Echo + ((i++)) + ((i<5)) + Read Line + read-u 9 + + curl-i-l-m 60-o/dev/null-s-w '%{http_code} ' http://111cn.net/ + Read Line + read-u 9 + + curl-i-l-m 60-o/dev/null-s-w '%{http_code} ' http://111cn.net/ + Read Line + read-u 9 + + curl-i-l-m 60-o/dev/null-s-w '%{http_code} ' http://111cn.net/ + Read Line + read-u 9 + + curl-i-l-m 60-o/dev/null-s-w '%{http_code} ' http://111cn.net/ + Read Line + read-u 9 + + curl-i-l-m 60-o/dev/null-s-w '%{http_code} ' http://111cn.net/ + Read Line + read-u 9 # FIFO file 5 empty rows read, wait for other child processes to write to FIFO + isok=200 + ' [' 200 = 200 '] ' + Echo Http://111cn.net/OK Http://111cn.net/OK + Echo # This subprocess completes the task, writes a blank line to the FIFO, initiates a child process + + curl-i-l-m 60-o/dev/null-s-w '%{http_code} ' http://111cn.net/ + Read Line + read-u 9 + isok=200 + ' [' 200 = 200 '] ' + Echo Http://50vip.com/OK Http://50vip.com/OK + Echo + + curl-i-l-m 60-o/dev/null-s-w '%{http_code} ' http://111cn.net/info/ + Read Line + read-u 9 + isok=200 + ' [' 200 = 200 '] ' + Echo Http://361a.net/OK Http://361a.net/OK + Echo + + curl-i-l-m 60-o/dev/null-s-w '%{http_code} ' http://111cn.net/ + Read Line + Wait # The URLs in the input file have been processed or processed in the subprocess, waiting for all child processes to finish + isok=200 + ' [' 200 = 200 '] ' + Echo Http://111cn.net/OK Http://111cn.net/OK + Echo + isok=000 + ' [' 000 = 200 '] ' + Echo http://5imovie.org/000 http://5imovie.org/000 + Echo + isok=200 + ' [' 200 = 200 '] ' + Echo Http://111cn.net/OK Http://52ixwebhosting.com/OK + Echo + isok=404 + ' [' 404 = 200 '] ' + Echo http://111cn.net/info/404 http://111cn.net/info/404 + Echo + isok=000 + ' [' 000 = 200 '] ' + Echo http://111cn.net/000 http://42.hcocoa.com/000 + Echo + echo $ ' 346211247350241214347273223346235237 ' Execution end + EXEC + Exit 0

Let's take a look at the following example

The code is as follows

Copy Code

#!/bin/bash function Pinghost { Ping $1-c 1-w |grep rtt|cut-d "/"-f6 } Tmp_fifofile= "/tmp/$.fifo" # script runs the current process ID number as filename Mkfifo $tmp _fifofile # Create a new random FIFO pipeline file exec 6<> $tmp _fifofile # definition file Descriptor 6 point to this FIFO pipe file RM $tmp _fifofile thread=10 For ((i=0;i< $thread i++));d O # for Loop writes 10 blank lines to the FIFO pipe file Echo Done >&6 While read domain Todo Read-u6 # Reads rows from the file descriptor 6 (actually pointing to the FIFO pipeline) { Pinghost ${domain}; # Execute Pinghost function Echo >&6 # Again writes a blank line to the FIFO pipeline file. }& # put in the background to perform DoneWait #因为之前的进程都是后台执行, so it is only after waiting that all the processes have been executed that the entire script runs out. EXEC 6>&-#删除文件描述符6 Exit 0

Description: {} This part of the statement is put into the background as a child process execution, this part is done almost simultaneously, when the 10 empty lines in the FIFO are read and the while Loop
continues to wait for read FIFO data, and when the 10 sub processes in the background are queued to FIFO, enter a blank line, So the FIFO has data, and the for statement continues to execute.