Shell's simple bulk Curl interface script

Shell script can be said to be very useful, in the server domain, using the shell to manipulate the transaction can be more convenient than the manual click is much faster. It's a text-only interface, but its powerful processing capabilities make all kinds of operations unimaginable. In addition, these habits can be transplanted into the daily work, improve efficiency.

In fact, shell syntax is very simple, basically is a combination of commands at the command line, a set of commands, and then formed a shell script. Of course, do not understand the grammar, Baidu search a bit better, after all, it is important to think rather than grammar.

Recently, a demand was just received, as follows:

The DBA will export the data of some service rules and then manually go to curl an application interface to complete the corresponding business requirements.

So the problem is, the DBA exports the data is formatted, to curl the interface is also formatted, need, just replace the corresponding data to the corresponding value. Note that there is no guarantee that all commands will succeed and you may need to run the interface again.

Obviously, it is feasible for a few data to write the Curl command manually, then execute it, then observe the result and make a judgment. But assuming that there are hundreds of, thousands of, tens of thousands of of the data, it is impossible to do it manually. So, the shell script is going to play (of course, some students say that I can in other languages, even say that my function is written in the code, but these special meaningless code, it does not need long-term retention).

The shell script can do just three things:

1. Read the contents of the source data file and replace the data format of the interface;

2. Execute the order to complete the business operation;

3. Record the complete log for later troubleshooting;

The demand is very simple, does not understand the grammar does not matter, checks. The reference code is as follows:

#!/bin/Bashlog_file='Result.log'Param_file=$1# source data specified log_cmd on the command line="tee-a $log _file"I=1 forLineinch`Cat$param _file '; Do   Echo "Read Line"$i":"$line |Tee-a $log _file let"i= $i +1"old_ifs= $IFS; ifs=","; Arr=($line) # segmenting data into an array IFS=$OLD _ifs; Curl_cmd="curl-d ' uid=${arr[0]}&bid=${arr[1]}&ba=${arr[2]}&to=6&bp=30&fddays=5 ' http://localhost : 8080/mi/api/ss/1.0.1/co/apply"   Echo`Date "+%y-%m-%d%h:%m:%s"`"Start ===>>"$curl _cmd |Tee-a $log _file eval"$curl _cmd 2>&1"|Tee-a $log _file # Use the eval command to bring back the error log and interface return results to the subsequent console and log storageEcho`Date "+%y-%m-%d%h:%m:%s"`"End <<==="$curl _cmd |Tee-a $log _file DoneEcho`Date "+%y-%m-%d%h:%m:%s"`"Over:end of Shell"|Tee-A $log _file

The source data format is referenced as follows:

234,201708222394083443,4211,201782937493274932, at201749379583475934,

The trick here is to use the tee command, which shows both the access log on the console and the records written to the file . There is manual observation, there is also log storage for viewing.

In this way, it is possible for us to make mistakes by manually tapping the data. The DBA exports formatted data from the data, and the shell script reads the formatted data directly, preserving the records. That's what the program does.

A word, try to steal a lazy, this is what we should do.

However, it should be noted that when an interface is executed by a script, you can consider concurrency problems, server pressure problems, and do not trust the code too much. To prepare for the worst.

Curl Command Please refer to: https://curl.haxx.se/docs/manpage.html (you can search briefly in Chinese description, of course)

In the past, I think 1, 2G of log file processing is a headache, but later found that grep, awk, sed, less, salt and other tools combined, can let you from dozens of G or even more of the mighty files, straight to the key. This is what Linux is all about.

Shell's simple bulk Curl interface script