Linux Shell Programming Practical skills

At present, more and more enterprise applications are deployed on Linux systems, and Linux Shell scripts can greatly help us to perform operational tasks for these applications. This makes the Linux Shell development skills an important and competitive skill for developers. Based on the author's actual development experience, this paper Korn Shell as an example to share the common problems in script development and related techniques.

Avoid common problems with timed task scripts

Many scripts are actually used in a timed task rather than manual operation. However, scripts that implement the same functionality may encounter different problems in both modes of operation.

Scripts that run as scheduled tasks often encounter the following issues.

• Path problem: The current directory is often not the directory where the script files reside. As a result, scripts cannot easily use relative paths when referencing external files that they use, such as configuration files and other script files.

• The command could not find a problem: Some external commands used in the script can be called normally when the script is executed manually. However, running under a timed task might have the problem that the script parser could not find the related command.

• The script repeats the problem: the execution of one script does not end, and the next time the script runs has started. Causes multiple processes in the system to run the same script at the same time.

Below are some of the common problems that are dealt with in the development of timed task scripts.

Path problem

The current path under a timed task is often not the directory where the script file resides. So we need to use absolute paths to refer to them. That is to get the directory where the script is located, and then use the directory as the basis for the absolute path to refer to the external files required by the script. method as shown in the following code.

Listing 1. Get the path to the script file

#!/usr/bin/kshecho "Current path is: ' pwd '" scriptpath= ' dirname $ #获取脚本所在路径echo "The script is located at: $scriptPath" ca T "$scriptPath/readme" #使用绝对路径引用外部文件

Place the script in Listing 1 under Directory/opt/demo/scripts/auto-task and add the script in cron. The scheduled task runs output as follows.

Current path is:/home/viscent

The script is located at:/opt/demo/scripts/auto-task

Command could not find the problem

A script that runs under a timed task may have an issue where the script parser cannot find the related command. For example, the Sqlplus command in an Oracle database, which is executed under a timed task without special handling when invoking the command, causes the script parser to not find the command, and an error message appears as follows:

Sqlplus:command not found

This is because the script is executed by a non-logon shell when it executes under a timed task, and the parent shell that executes the script is not the Oracle user's shell. Therefore, the Oracle user's. profile file is not called at this time. So the workaround is to add the following code at the beginning of the script:

Listing 2. Resolve external command issues not found

Source/home/oracle/.profile

It also says that for problems that cannot be found with external commands, it can be resolved by adding a statement to the source user's. profile file at the beginning of the script.

Script repeat Run issues

Another common problem with timed task scripts is the repeated run of scripts. For example, a script is set to run every 5 minutes. If the script does not end in 5 minutes at a time, the scheduled task service will still start a new process to execute the script. At this point, multiple processes running the same script appear. This can lead to scripting dysfunction. And it wastes system resources. There are usually two ways to avoid a script from running repeatedly. The first is to check if the system has other processes running the script when the script executes. If present, terminates the current script's run. Second, the script runs to check if there are other processes running the script in the system. If there is, end the process (this method has some risks, use with caution!) ）。 Both methods need to check the beginning of the script to see if the system already has a process running the current script, and if so, get the PID of the process. The sample code is shown in Listing 3 below.

Listing 3. Prevent scripts from running repeatedly method 1

#!/usr/bin/kshmain () {selfpid= "$$" scriptfile= "$" typeset existingpidexistingpid= ' getexistingpids $selfPID "$ ScriptFile "' If [!-Z" $existingPid "]; Then echo "The script already running, exiting ..." Exit-1fidoitstask} #获取除本身进程以外其它运行当前脚本的进程的 Pidgetexistingpids () {SELFP Id= "$" scriptfile= "$" ps-ef | grep "/usr/bin/ksh ${scriptfile}" | Grep-v "grep" | awk "{if (\$2!= $selfPID) print \$2}"}doitstask () {echo "task is now being executed ..." sleep #睡眠 20s to simulate a script that takes a long time to complete a task }main $*

Listing 4. Prevent scripts from running repeatedly method 2

#!/usr/bin/kshmain () {selfpid= "$$" scriptfile= "$" typeset existingpidexistingpid= ' getExistingPIDs $ selfpid  "$scriptFile" ' if [ ! -z  ' $existingPid " ]; then  echo   "The script already running, killing it ..."   kill -9  "$ Existingpid "  #此方法有一定风险, use with caution! Fidoitstask} #获取除本身进程以外其它运行当前脚本的进程的  pidgetexistingpids () {selfpid= "$" scriptfile= "$" ps -ef |  grep  "/usr/bin/ksh ${scriptfile}"  | grep -v  "grep"  | awk  "{  if (\$2!= $selfPID)  print \$2 } "}doitstask () {echo " task is now being  executed. "sleep 20   #睡眠  20s to simulate a script that takes a long time to complete a task}main $*

Back to top of page

Script Debugging Tips

Although a common problem with Shell development is the difficulty of debugging, the lack of effective debugging tools. However, we can take some steps to help us avoid debugging difficulties in the way of development and debugging. Because it is a script development, many people are accustomed to write code from a line, a script even a function is not. Although this approach is not problematic in syntax and functionality. But this increases the difficulty of debugging. Conversely, if you use a modular approach to scripting, it makes the code structure clear and easy to debug. This can be seen in an example.

Assume that the following script is capable of collecting related log files in a production environment for locating problems. The log files that need to be collected include the operating system log, the middleware log, and the log of the application system itself. These files are compressed into a GZ file.

Listing 5. Automatic collection of log files

#!/usr/bin/kshmain () {collectsyslog #收集系统日志文件collectMiddlewareLog #收集中间件日志文件collectAppLog #收集应用系统日志文件tar-ZCF Logs.tgz syslog.zip mdwlog.zip applog.zip #将三中类型的日志打包, easy to download}

If the script execution reported the following error:

Tar:applog.zip:Cannot stat:no such file or directory

We can quickly lock the Collectapplog function. Because it is responsible for outputting applog.zip this file. There is no need to look at other parts of the code.

Another benefit of adopting a modular approach is that the results of code debugging can be consolidated. For example, if you have debugged the function that the operation State Log collects. When debugging other functions, the code that is being debugged may need to be changed. However, these changes may not be large enough to affect the code that was previously debugged. Conversely, if a script is a statement throughout, without a function, one line of code is changed, and the ability to collect three kinds of logs may be affected.

Another typical scenario is that during scripting, we might write some tentative code because we're not sure how to handle some of the problems. Then, through the repeated debugging to confirm the correct processing mode. In fact, these tried-and-true code might be a statement or even a command. But a lot of people are in the large section of the code repeatedly to debug this small piece of code. This will be very time consuming. Especially when debugging errors in other parts of the code during debugging, the author has to resolve other errors first, otherwise the code we really want to debug cannot be executed. In this case, write a small script specifically for testing.

Debugging in this script also we are not sure how to write the code, how to "integrate" it into the script we are developing. This improves debugging efficiency and avoids the time that should not be consumed. For example, we need to get the process ID of the process in which the script itself is written. At this point, we're not quite sure how to write the code that gets the current process ID. Then, we can create a new test script in which to try to implement the function of getting the process ID. After finding the right method, "port" the code into the script that we really want to develop.

Back to top of page

Processing large segment character output

One of the most frequently asked questions in script development is the output hint message. Of course, using the echo command is sufficient for a short message output. However, it is not elegant to use the echo command for large segments of the message output. A more appropriate approach is to use the Cat command in conjunction with input redirection. This is illustrated by a specific example below.

Assume that the following script installs a program into a user-specified directory. If the user-specified directory does not exist, the prompt

The user checks that the specified directory is correct and executes the script again.

Listing 6. Use the echo command to output a large segment of characters

#!/usr/bin/kshpath= "$" if [ ! -d  "$path"  ]; then          #这里还必需处理星号这个特殊字符的显示echo   ' **************************************************** '   echo errorecho  "The destination directory not exists,make sure below  directory you specified is correct: "echo ${path}echo " Then re-run  this script. " echo  ' **************************************************** ' fi 

The code in this way is not very readable, and the reader needs to read multiple echo commands and then "synthesize" to understand exactly what the cue message is. Also, once the information is prompted to change. This change may result in syntax errors due to a special character such as a double quotation mark being accidentally changed when one of the echo commands is altered, which affects the execution of the entire script.

The code in Listing 7 shows how to better handle the output of large pieces of text using cat commands and input redirection.

Listing 7. Output a large segment of characters using the cat command

#!/usr/bin/kshpath= "$" if [!-D "$path"]; Thencat<<eof****************************************************errorthe destination directory not exists, Make sure belowdirectory specified are correct:${path}then re-run this script.************************************** Eoffi

Obviously, the code for this approach is more concise and more readable. Readers only need to see a single command, they know the specific content of the message. Also, to modify the prompt, we can safely change the portion between the two file Terminator EOF. Even if the modification is wrong, it will not affect the rest of the code.

Back to top of page

Avoid the use of non-essential temporary files

Beginners often use temporary files when writing Shell scripts without having to use temporary files. This not only increases the amount of code written (used to handle creating, reading, and deleting temporary files, etc.), but it may make the script run slower (file I/O is not a fast operation after all).

The following example assumes that the function of a script is to add the following line of text to all the. txt files in the current directory:

--end of file name--

The code in Listing 8 and listing 9 shows the code that uses temporary files without having to use temporary files, and code that does not require temporary files.

Listing 8. Use temporary files without having to use temporary files

#!/usr/bin/kshls-lt *.txt | awk ' {print $NF} ' > tmp #将命令输出重定向到临时文件 tmpcat tmptypeset filenametypeset lastlinewhile read FileName #逐行读取临时文件中的每一行do l Astline= ' tail-1 "$fileName" ' if [! "$lastLine" = = "--end of $fileName--"]; Then echo "--end of $fileName--" >> $fileName fidone <tmp #从临时文件进行输入重定向rm tmp #删除临时文件

Listing 9. Do not use temporary files

#!/usr/bin/kshtypeset Filenametypeset lastlinefor fileName in $ (ls-lt *.txt | awk ' {print $NF} ') do lastline= ' Tail-1 ' $fi Lename "' If [! "$lastLine" = = "--end of $fileName--"]; Then echo "--end of $fileName--" >> $fileName Fidone

This article from "Passer" blog, declined reprint!