Practical skills in LinuxShell programming

At present, more and more enterprise applications are deployed on Linux systems, and the LinuxShell script can greatly help us complete the O & M tasks of these applications. This makes LinuxShell development skills an important and competitive skill for developers. This article focuses on the author's Linux skills

At present, more and more enterprise applications are deployed on Linux systems, and Linux Shell scripts can greatly help us complete the O & M tasks of these applications. This makes Linux Shell development skills an important and competitive skill for developers. Based on the actual development experience of the author, this article uses the Korn Shell as an example to share common issues and related skills in script development.

Avoid regular task scripts

Many scripts are usually run as scheduled tasks rather than manually. However, scripts that implement the same function may encounter different problems in these two running modes.

The following problems often occur when running scripts with scheduled tasks.

• Path problem: the current directory is often not the directory where the script file is located. Therefore, when a script references its external files, such as configuration files and other script files, it is not convenient to use the relative path.
• Command cannot be found: some external commands used in the script can be called normally when the script is manually executed. However, when running a scheduled task, the script parser may fail to find the relevant commands.
• Duplicate script running: the execution of a script has not ended, and the next script has started. As a result, multiple processes in the system run the same script at the same time.

The following describes how to solve the preceding common problems in the script development of a scheduled task.

Path Problems

The current path of a scheduled task is often not the directory where the script file is located. Therefore, we need to use absolute paths for reference. That is, first obtain the directory where the script is located, and then use the absolute path based on the directory to reference the external files required by the script. The method is shown in the following code.

　　Listing 1. obtain the path of the script file

#! /Usr/bin/kshecho "Current path is: 'pwd'" scriptPath = 'dirname $ 0' # obtain The script path echo "The script is located: $ scriptPath "cat" $ scriptPath/readme "# Use an absolute path to reference an external file

Place the script in listing 1 under the Directory/opt/demo/scripts/auto-task and add the script in cron. The scheduled task running output is as follows.

Current path is:/home/viscent

The script is located at:/opt/demo/scripts/auto-task

Command cannot be found

The script running in the scheduled task may fail to find the relevant commands in the script parser. For example, the sqlplus command in the Oracle database, if the script does not have special processing when calling this command, it will be executed in the scheduled task, the script parser will not be able to find this command, the following error message is displayed:

Sqlplus: command not found

This is because the script is executed in a scheduled task by a non-logon Shell, and the parent Shell for executing the script is not the Shell of the Oracle user. Therefore, the. profile file of the Oracle User is not called. The solution is to add the following code at the beginning of the script:

　　Listing 2. solving the problem that external commands cannot be found

source /home/oracle/.profile

That is to say, you can add a source user's. profile File statement at the beginning of the script to solve the problem that external commands cannot be found.

Repeated script running

Another common problem with scheduled task scripts is repeated running of scripts. For example, a script is set to run every 5 minutes. If a script cannot be run within five minutes, the scheduled task service starts a new process to execute the script. In this case, multiple processes running the same script are displayed. This may cause disorder in script functions. And system resources are wasted. There are usually two methods to avoid repeated running of scripts. First, check whether other processes running the script exist in the system during script execution. If yes, the current script is terminated. Second, check whether other processes in the system run the script during script execution. If yes, end the process. (This method is risky. use it with caution !). Both methods need to check whether the system already has a process running the current script at the beginning of the script. if such a process exists, obtain the PID of the process. The sample code is shown in listing 3.

　　Listing 3. preventing repeated running of scripts method 1

#! /Usr/bin/kshmain () {selfPID = "$" scriptFile = "$0" typeset existingPidexistingPid = 'getexistingpids $ selfPID "$ scriptFile" 'if [! -Z "$ existingPid"]; then echo "The script already running, exiting... "exit-1 fidoItsTask} # obtain the PIDgetExistingPIDs () of the processes running the current script except the processes themselves () {selfPID = "$1" scriptFile = "$2" ps-ef | grep "/usr/bin/ksh $ {scriptFile}" | grep-v "grep" | awk" {if (\ $2! = $ SelfPID) print \ $2} "} doItsTask () {echo" Task is now being executed... "sleep 20 # sleep for 20 s to simulate a script to execute a task that requires a long time} main $ *

　　Listing 4. preventing repeated running of scripts method 2

#! /Usr/bin/kshmain () {selfPID = "$" scriptFile = "$0" typeset existingPidexistingPid = 'getexistingpids $ selfPID "$ scriptFile" 'if [! -Z "$ existingPid"]; then echo "The script already running, killing it..." kill-9 "$ existingPid" # this method is risky. use it with caution! FidoItsTask} # obtain the PIDgetExistingPIDs () of the processes running the current script other than the process itself () {selfPID = "$1" scriptFile = "$2" ps-ef | grep "/usr/bin/ksh $ {scriptFile}" | grep-v "grep" | awk" {if (\ $2! = $ SelfPID) print \ $2} "} doItsTask () {echo" Task is now being executed... "sleep 20 # sleep for 20 s to simulate a script to execute a task that requires a long time} main $ *

Script debugging skills

Although a common problem in Shell Development is debugging difficulties, there is a lack of effective debugging tools. However, we can adopt some development and debugging methods that can help us avoid debugging difficulties to a certain extent. Because it is script development, many people are used to writing code directly in a line. there is no function in a script. Although this method has no syntax or function problems. However, this increases the difficulty of debugging. On the contrary, if you use a modular method to write scripts, the code structure is clear and easy to debug. This is an example.

Assume that the following script is used to collect related log files in the production environment and locate the problem. The log files to be collected include operating system logs, middleware logs, and application system logs. These files are compressed into a gz file.

　　Listing 5. automatically collecting log files

#! /Usr/bin/kshmain () {collectSyslog # collect the system log file collectMiddlewareLog # collect the middleware log file collectAppLog # collect the application system log file tar-zcf logs. tgz syslog.zip mdwlog.zip applog.zip # pack the three types of logs for easy download}

If the following error is reported during script execution:

Tar: applog.zip: Cannot stat: No such file or directory

We can quickly lock the function collectAppLog. Because it outputs the applog.zip file. There is no need to look at other parts of the code.

Another advantage of the modular approach is that the code debugging results can be consolidated. For example, in the above example, if you have already debugged the function for collecting operation status logs. When debugging other functions, the code to be debugged may need to be modified. However, these changes may not affect the code that has been debugged before. On the contrary, if a script contains all statements without functions, one line of code may be modified, and the three log collection functions may be affected.

Another typical scenario is that during script writing, we may write some tentative code because we are not sure how to solve some problems. Then, the correct processing method is confirmed through repeated debugging. In fact, these tentative code may be a statement or even a command. However, many people repeatedly debug the code in large sections. This will consume a lot of time. Especially when an error occurs when debugging other parts of the code during the debugging process, the author must first solve other errors. Otherwise, the code that we really want to debug may not be executed. In this case, a small test script is specially written.

We are not sure how to write the code for debugging in this script, and how to "integrate" it into the script we are developing. This improves debugging efficiency and avoids unnecessary time consumption. For example, we need to obtain the process ID of the process where the script is located during writing. At this time, we are not sure how to write this code to obtain the current process id. Then, we can create a new testing script in which we can try to obtain the process ID. Find the correct method and "port" the code to the script we actually want to develop.

Process output of large characters

A common problem to be dealt with during script development is the output of prompt information. Of course, it is sufficient to use the echo command to output brief prompts. However, it is not elegant to use the echo command for output of large-segment prompt information. A more suitable method is to use cat commands in combination with input redirection. Here is a specific example to illustrate this.

Assume that the following script installs a program to a specified directory. If the specified directory does not exist, the system prompts

The user checks whether the specified directory is correct and re-executes the script.

　　Listing 6. output large characters using the echo command

#! /Usr/bin/kshpath = "$1" if [! -D "$ path"]; then # Here, you must also process the display echo of the special character asterisk '************************** * ************************ 'echo ERRORecho "The destination directory not exists, make sure below directory you specified is correct: "echo $ {path} echo" Then re-run this script. "echo '************************************* * *************** 'Fi

The code in this way is not very readable. readers need to read multiple echo commands and then perform "synthesis" to understand the prompts accurately. In addition, you need to modify the information once prompted. This change may cause a syntax error because a special character such as double quotation marks is accidentally added when an echo command is modified, thus affecting the execution of the entire script.

The code in listing 7 shows how to use cat commands and input redirection to better process output of large text segments.

　　Listing 7. use the cat command to output large characters

#! /Usr/bin/kshpath = "$1" if [! -D "$ path"]; thencat <
 
  
Obviously, the code in this processing method is more concise and easier to read. Readers only need to read a command to know the specific content of the prompt information. In addition, if you want to modify the prompt, you can safely change the part between the two file terminator EOF. Even if the modification is wrong, it will not affect other parts of the code.
Avoid unnecessary temporary files
  
When writing Shell scripts, beginners often use temporary files without the need to use temporary files. This not only adds additional coding workload (used to process the creation, reading, and deletion of temporary files ), in addition, the script may run slowly (file I/O is not a fast operation after all ).
  
The following example assumes that a script is used to add the following line of text to all the. txt files in the current directory:
  
-EndFile name-
  
The code in listing 8. and listing 9. respectively shows the code for using a temporary file and the code for using a temporary file without any need.
  
　　Listing 8. using temporary files without having to use temporary files
  
#! /Usr/bin/kshls-lt *. txt | awk '{print $ NF}'> tmp # redirects the command output to the temporary file tmpcat tmptypeset fileNametypeset lastLinewhile read fileName # read each line of the temporary file row by row lastLine = 'tail -1 "$ fileName" 'if [! "$ LastLine" = "-- End of $ fileName --"]; then echo "-- End of $ fileName --"> $ fileName fidone
   
    
　　Listing 9. do not use temporary files
    
#!/usr/bin/kshtypeset fileNametypeset lastLinefor fileName in $(ls -lt *.txt | awk '{print $NF}')do lastLine=`tail -1 "$fileName"` if [ ! "$lastLine" == "--End of $fileName--" ]; then   echo "--End of $fileName--" >> $fileName fidone
Use an editor that supports FTP
    
If your development environment is in a Windows operating system, and the test is performed on Linux through the terminal software (such as Putty. In this case, many developers are used to directly edit scripts (such as using vi commands) on the terminal software ). Obviously, this method is inefficient in editing. In addition, script development usually requires modifying and testing. Even if it is a syntax error, due to lack of tool support, we may need to run the script to find out. Therefore, improving the efficiency of script editing improves the development efficiency to some extent. A good option to improve script editing efficiency when developing scripts on Windows systems is to use an editor that supports simple FTP functions, such as Editplus and UltraEditor. You can use these editors to "open" (actually download) the script files on the Linux testing host using FTP. When the script is edited and saved, the editor automatically uploads the script to the test host. Next, you only need to test the script through the terminal software. If the script needs to be modified after the test, you can use the "reload document" function of the editor (you can usually set shortcuts for this function ).