How to implement Shell concurrency

Reprint: http://blog.chinaunix.net/uid-27571599-id-3473078.html

Many people have asked me how to write shell script, how to achieve the same time to upload files to three FTP servers, how to detect the three servers are alive and so on, in fact, this is to realize the shell concurrency.     So how does the shell concurrency be implemented? Let me take this example:
Each task is the output character "BINGFA" and stays for a second, a total of 20 times.
In normal thinking, the script should write this:
[Root@station1 ~]# cat a.sh #!/bin/bash for ((i=0;i<20;i++)) did sleep 1 echo "BINGFA" done [Root@station1 ~]# time Bash a.sh bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingf A BINGFA bingfa bingfa real 0m20.067s user 0m0.016s sys 0m0.031s [root@station1 ~]# can see that it took about 20 seconds to execute this script. So how to write with Shell concurrency, many people will think of background programs, similar to the following:
[Root@station1 ~]# cat b.sh #!/bin/bash for ((i=0;i<20;i++)) Doing {Sleep 1 echo ' BINGFA '}& doing wait [root@station1 ~]# time bash b.sh bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingfa bingf A bingfa bingfa bingfa bingfa bingfa real 0m1.060s user 0m0.005s sys 0m0.057s [root@station1 ~]#
It takes about a second to write, and you can see that all the tasks are executed almost simultaneously, if the task is very large, the system will not be able to withstand, but also affect the operation of other programs in the system, which requires a control of the number of threads. Here's the code I wrote at the beginning (it's problematic):

[Root@station1 ~]# cat c.sh #!/bin/bash exec 6<>tmpfile echo "1\n1\n1" &>6 for ((i=0;i<20;i++)) do Read-u 6 {Sleep 1 echo "$REPLY" echo "1" 1>&6}& do wait [root@station1 ~]# time Bash c.sh 111 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 real 0m1.074s user 0m0.012s sys 0m0.031s [root@station1 ~]#
It is obvious that there is a problem, I would like to control the number of threads is 3, but even if the file descriptor 6 is empty, it will be read empty, and then skip the execution below, so using the file descriptor to open a file is no good, and then I think about using a similar pipe file to do, here is my code:

[Root@station1 ~]# cat d.sh #!/bin/bash mkfifo fd2 exec 9<>fd2 echo-n-E "1\n1\n1\n" 1>&9 for ((i=0;i<20;i + +) do read-u 9 {#your process sleep 1 echo "$REPLY" Echo-ne "1\n" 1>&9} & did wait rm-f fd2 [Root@stati On1 ~]# Time Bash d.sh 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 The real 0m7.075s user 0m0.018s sys 0m0.044s [root@station1 ~]# So OK, three threads running 20 tasks, 7 seconds more points.