Use Shell scripts to generate XML file instances

Today, I sorted out a small case of my work after studying shell in this period and shared it with you! Coincidentally, as a Cainiao who just graduated for half a year, after entering the company, I listened to the recommendation of the company's Daniel to learn linux -"...

Today, I sorted out a small case of my work after studying shell in this period and shared it with you!

Coincidentally, as a rookie who just graduated for half a year, after entering the company, I listened to the recommendation of the company's Daniel to learn linux-"laruence's private food", basically, it started from July to July of this year. Basically, I read the basics and started to understand the knowledge of shell scripts. The company gave me a shell script case, and the time was not too tight. Then I started learning and started to do it. although the customer had been delayed for two weeks to reflect the previous business logic problem, it was finally finished and I was very happy to learn what I learned, I am idle today. I sorted out the code and shared it with you.

The details are as follows:

The requirement is to write a shell script, install and query the data, assemble the qualified data according to the xml style specified by the customer, and then add it to the crontab, scheduled execution is put on the client server through scp or ftp.

Specific steps:

I. compile the code for generating the xml document

#! /Bin/bash # filename: create_xml.sh # create_wangxb_20150123 # use the first parameter passed in from the outside as the xml file name outfile =1 1 # Indent tabs = 0 # ++ ++ # assemble a node, output to file # Let's talk about the differences when passing parameters: if there is a command executed by the following script #/path/to/scriptname opt1 opt2 opt3 opt4 # $0: the value is the name of the script by default. starting from $1-$4, it is the value of the parameter ## #: indicates the "number" of the subsequent parameter @: represents "" $1 "" $2 "" $3 "" $4 "". each variable is independent (enclosed in double quotation marks); # $ *: indicates "" $ 1c $ 2c $ 3c $4 "", where c is the separator byte. the default value is the blank key, therefore, this example indicates "$1 $2 $3 $4. # In shell, you can also use $ {} to include the variable name, to call the variable # ++ put () {echo '<' $ {*} '> $ outfile} # Here is also a node that outputs xml, it only has more settings than the above node # $ {@: 2} means that its value is from the second parameter to the last parameter. why? Sometimes your second parameter may contain spaces. shell accepts the put_tag () {echo '<' $1 '>' $ {@: 2} 'calculated by space }'
 '>>> $ Outfile} # It is also an output node function, but CDATA is added to prevent xml parsing failure caused by special characters: put_tag_cdata () {echo' <'$1'>&#39;${@:2}&#39;
 '>>> $ Outfile} put_head () {put '? '$ {1 }'? '} # This is an indent algorithm. you can understand out_tabs () {tmp = 0 tabsstr = "" while [$ tmp-lt $ (tabs)] do tabsstr =$ {tabsstr} '\ t' tmp = $ (tmp + 1) done echo-e-n $ tabsstr >>>$ outfile} tag_start () {out_tabs put $1 tabs =$ (tabs + 1)} tag () {out_tabs if ["$1" = 0] then put_tag $2 $ (echo $ {@: 3 }) elif ["$1" = 1] then put_tag_cdata $2 $ (echo $ {@: 3}) fi} tag_end () {tabs = $(tabs-1 )) out_tabs put '/' ${1 }}

Here are some basic knowledge:

Parameters:

Assume that the following script executes the command:
/Path/to/scriptname opt1 opt2 opt3 opt4

$0: The value is the script name by default. from $1-$4, it is the parameter value $ #: indicates the "number" of the subsequent parameter. $ @: represents "" $1 "" $2 "" $3 "" $4 "". each variable is independent (enclosed in double quotation marks). $ *: indicates "" $ 1c $ 2c $ 3c $4 "", where c is the separator byte. the default value is the blank key, therefore, this example indicates "$1 $2 $3 $4. In shell, we can also use $ {} to contain the variable name to call the variable.

2. query data from the database and use the above functions to create an xml file

#! /Bin/bash # filename: ts_xml.sh # create_wangxb_20150126 # PATH =/u01/app/oracle/product/10.2.0/db_1/bin:/usr/kerberos/bin: /usr/local/bin:/usr/bin:/opt/dell/srvadmin/bin:/home/p3s_batch/tools: /home/p3s_batch/binexport PATH # Database account information filesource ~ /. P3src # ++ ++ # set some variable # XMLSCRIPT: absolute path of the script # MATCHING_RESULT_XML: xml_1 file name # XML_FUNC_FILE: generate the xml function file path # MATCHING_RESULT_QUERY_DATA: sqlplus find the zero-time file for data storage # MATCHING_RESULT_QUERY_ SQL: SQL statement # ++ ++ # below are some basic settings for export XMLSCRIPT =/usr/p3s/batch/jaaa_match /tmp_xa_wangxbXML_DIR = "$ XMLSCRI PT/xmldata "XML_FUNC_FILE =" xml_func.sh "MATCHING_RESULT_XML =" matching_result _ "$ (date '+ % Y % m % d _ % H % M % S ')". xml "MATCHING_RESULT_QUERY_DATA =" matching_result_query_data.tmp "placement =" matching_result_query. SQL "CLIENT_LIST_XML =" client_list _ "$ (date '+ % Y % m % M % d _ % H % m % S ')". xml "CLIENT_LIST_QUERY_DATA =" client_list_query_data.tmp "CLIENT_LIST_QUERY_ SQL =" client_list_query. SQL "# add_wangxb_20150225if [! -D "$ XML_DIR"]; then mkdir $ XML_DIRfi # ++ # modify_wangxb_201501_# check for temporary file # ++ if [-e "$ XML_DIR/$ MATCHING_RESULT_XML"]; then rm-f $ XML_DIR/$ MATCHING_RESULT_XMLfiif [-e "$ XMLSCRIPT/$ MATCHING_RESULT_QUERY_DATA"]; then MATCHING_RESULT_QUERY_DATA = "matching_result_query_data _" $ (date '+ % Y % m % d % H % M % S ')". tmp "fi # ++ ++ # Add_wangxb_20150225 # check system time, choice query time period # This is based on the time that crontab executes on a daily basis, obtain the time range of the where condition when we query the database # ++ +++ ++ sys_datetime =$ (date '+ % Y % m % d % h ') first_chk_datetime = "$ (date '+ % Y % m % d') 04" second_chk_datetime = "$ (date' + % Y % m % d ') 12 "third_chk_datetime =" $ (date '+ % Y % m % d') 20 "# because the crontab of the server is the time above, but the shell is executed a lot, when calling this shell, it is not necessarily, 12: 30, so, here, the time range is relatively wide based on the system time. case $ sys_datetime in "$ first_chk_datetime" | "$ (date '+ % Y % m % d ') 05 "|" $ (date '+ % Y % m % d') 06 "|" $ (date' + % Y % m % d') 07 ") chk_start = $ (date '+ % Y-% m-% d 21:00:00'-D'-1 DAY ') chk_end = $ (date '+ % Y-% m-% d 04:29:59'); "$ second_chk_datetime" | "$ (date '+ % Y % m % d ') 13 "|" $ (date '+ % Y % m % d') 14 "|" $ (date' + % Y % m % d') 15 ") chk_start = $ (date '+ % Y-% m-% d 04:30:00') chk_end = $ (date '+ % Y-% m-% d 12:29:59 '); "$ Third_chk_datetime" | "$ (date '+ % Y % m % d') 21" | "$ (date' + % Y % m % d ') 22 "|" $ (date '+ % Y % m % d') 23 ") chk_start = $ (date' + % Y-% m-% d 12:30:00 ') chk_end = $ (date '+ % Y-% m-% d 20:59:59'); *) chk_start = $ (date '+ % Y-% m-% d 00:00:00 ') chk_end = $ (date '+ % Y-% m-% d 23:59:59'); esac # modify_wangxb_20150310 # Test the oracle database connection. if the connection fails, subsequent code is no longer executed and the error log $ ORACLE_HOME/bin/sqlplus-s $ ORAUSER_WEB_PASDB <EOFset echo offset fe Edback offalter session set nls_date_format = 'yyyy-MM-DD: HH24: MI: SS'; select sysdate from dual; quitEOFif [$? -Ne 0] then echo "*********** DB has been stolen. **********" exitelse echo "********************" fi # sqlplus is a client software of oracle., for more information, see du Niang. here we will introduce the SQL statements and parameters to be executed, output the result to the specified file $ ORACLE_HOME/bin/sqlplus-s $ ORAUSER_WEB_PASDB @ $ XMLSCRIPT/$ success "$ chk_start" "$ chk_end"> $ XMLSCRIPT/$ MATCHING_RESULT_QUERY_DATA # create matching result's xml file # add_wangxb_20150227 # The following algorithm analyzes the detected data, call the xml function to generate the xml file source "$ XMLSCRIPT/$ XML_FUNC_FILE" "$ XML_DIR/$ MATCHING_RESULT_XML" put_head 'XML version = "1.0" encoding = "UTF-8" 'tag _ start' ROOT 'If [-s "$ XMLSCRIPT/$ MATCHING_RESULT_QUERY_DATA"]; then datas =$ {XMLSCRIPT}/$ {MATCHING_RESULT_QUERY_DATA} # for res in $ datas while read res; do stock_id = $ (echo $ res | awk 'In in {FS = "\\^\\ * \^" }{ print $1 }') seirino = $ (echo $ res | awk 'In in {FS = "\\^\\ * \^" }{ print $2 }') match_flg = $ (echo $ res | awk 'In in {FS = "\\^\\ * \^" }{ print $3 }') unmatch_riyuu = $ (echo $ res | awk 'In in {FS = "\\^\\ * \^" }{ print $4 }') up_date_tmp = $ (echo $ res | awk 'In in {FS = "\\^\\ * \^" }{ print $5 }') up_date = $ (echo $ up_date_tmp | awk 'In in {FS = "@"} {print $1 "" $2 }') tag_start 'matching 'tag 0 'stockid' $ {stock_id:-""} tag 0 'seirino' $ {SEIRINO:-""} tag 0 'result' $ {match_flg: -""} tag 1 'reason '$ {unmatch_riyuu:-""} tag 0 'update _ date' $ {up_date: -""} tag_end 'matching 'done <$ datasfitag_end 'root' rm $ XMLSCRIPT/$ MATCHING_RESULT_QUERY_DATA # create client list's xml file # add_wangxb_2015027 # generate an xml file, like above, if [-e "$ XML_DIR/$ CLIENT_LIST_XML"]; then rm-f $ XML_DIR/$ CLIENT_LIST_XMLfiif [-e "$ XMLSCRIPT/$ CLIENT_LIST_QUERY_DATA"]; then CLIENT_LIST_QUERY_DATA = "client_list_query_data _" $ (date '+ % Y % m % d % H % M % S ')". tmp "fi $ ORACLE_HOME/bin/sqlplus-s $ ORAUSER_MND @ $ XMLSCRIPT/$ response> $ XMLSCRIPT/$ response" $ XMLSCRIPT/$ XML_FUNC_FILE "" $ XML_DIR/$ CLIENT_LIST_XML "put_head 'XML version = "1.0" encoding = "UTF-8" 'tag _ start' root' if [-s "$ XMLSCRIPT/$ CLIENT_LIST_QUERY_DATA"]; then datas =$ {XMLSCRIPT}/$ {CLIENT_LIST_QUERY_DATA} # for res in $ datas while read res; do configuration_id = $ (echo $ res | awk 'In in {FS = "\\^\\ * \^" }{ print $1 }') configuration_name = $ (echo $ res | awk 'In in {FS = "\\^\\ * \^" }{ print $2 }') client_id = $ (echo $ res | awk 'In in {FS = "\\^\\ * \^" }{ print $3 }') client_print_name = $ (echo $ res | awk 'In in {FS = "\\^\\ * \^" }{ print $4 }') tag_start 'client' tag 0' CORPORATION _ id' $ {configuration_id:-""} tag 1 'Corporation _ name' $ {configuration_name: -""} tag 0 'Client _ id' $ {client_id:-""} tag 1 'Client _ PRINT_NAME '$ {client_print_name: -""} tag_end 'client' done <$ datasfitag_end 'root' rm $ XMLSCRIPT/$ CLIENT_LIST_QUERY_DATA # add_wangxb_20150304 # Convert xml file encoding # This is to transcode the xml file, command is iconvif [-e "$ XML_DIR/$ MATCHING_RESULT_XML"]; then echo "************ matching_result.xml ***************" iconv-f euc-jp-t UTF-8 $ XML_DIR/$ MATCHING_RESULT_XML-o $ XML_DIR/$ MATCHING_RESULT_XML.utf-8 mv $ XML_DIR/$ MATCHING_RESULT_XML.utf-8 $ XML_DIR/$ Export [-e "$ XML_DIR/$ CLIENT_LIST_XML"]; then echo "*********** client_list.xml **************" iconv-f euc -jp-t UTF-8 $ XML_DIR/$ CLIENT_LIST_XML-o $ XML_DIR/$ CLIENT_LIST_XML.utf-8 mv $ XML_DIR/$ CLIENT_LIST_XML.utf-8 $ XML_DIR/$ CLIENT_LIST_XMLfi # add_wangxb_20150304 # Send the xml file to the destination server ftp # ftp_host= "222. ***. ***. * ** "# USER =" *** "# PASS =" *** "# ftp-I-n $ ftp_host <EOF # user $ USER $ PASS # cd/ # LCD $ XML_DIR/# put $ MATCHING_RESULT_XML # put $ CLIENT_LIST_XML # quit # EOF # test ftp # put the xml file on the client server through ftp, ftp_host: client server address, user login name, pass password ftp_host = "***. ***. ***. * ** "USER =" *** "PASS =" *** "dir ="/upload "ftp-I-n $ ftp_host <EOFuser $ USER $ PASScd/upload /LCD $ XML_DIR/put $ MATCHING_RESULT_XMLput $ CLIENT_LIST_XMLquitEOF # Save the program log fileYYMM = $ (date + '% Y % m % d % H % M ') cp/tmp/create_xml.log/usr/p3s/batch/jaaa_match/tmp_xa_wangxb/logs/create_xml.log. $ YYMM # Send error log files into the Admin mailboxinfo_to_mail_1 = "** @ ** .co.jp" info_to_mail_2 = "*** @ ** .co.jp" # nkf Japanese transcoding command title = $ (echo "test" | nkf-j) nkf-j </tmp/create_xml.log | mail-s $ title $ info_to_mail_1 $ info_to_mail_2 # exit

It was originally transmitted using scp,But it is modified later. here we will find one of the ssh users who can log in immediately without a password.

The following two SQL files are executed:

SET PAGESIZE 0SET FEEDBACK OFFSET VERIFY OFFSET ECHO OFFSET HEADING OFFSET TIMI OFFSET LINESIZE 1000SET WRAP OFFSELECT s.STOCKID|| '^*^' ||a.SERI_NO|| '^*^' ||a.MATCH_FLG|| '^*^' ||a.UNMATCH_RIYUU|| '^*^' ||to_char(a.UP_DATE,[email protected]:MI:SS') UP_DATE FROM aaa_stock_db a LEFT JOIN SENDDATAAPPRAISALPROTO s ON a.SERI_NO=s.SEIRINO WHERE a.UP_DATE BETWEEN to_date('&1','yyyy-mm-dd hh24:mi:ss') AND to_date('&2','yyyy-mm-dd hh24:mi:ss') AND a.DEL_FLG=0 ORDER BY a.UP_DATE DESC;exit

SET PAGESIZE 0SET FEEDBACK OFFSET VERIFY OFFSET ECHO OFFSET HEADING OFFSET TIMI OFFSET LINESIZE 1000SET WRAP OFFSELECT a.CORPORATION_ID|| '^*^' ||a.CORPORATION_NAME|| '^*^' ||b.CLIENT_ID|| '^*^' ||(select CLIENT_PRINT_NAME from CLIENT_MASTER where CLIENT_ID = b.CLIENT_ID) as CLIENT_PRINT_NAME FROM M_CORPORATION_MASTER a LEFT JOIN M_CORPORATION_GROUP b ON (a.CORPORATION_ID = b.CORPORATION_ID) WHERE a.DEL_FLG=0 AND b.DEL_FLG=0;exit

3. let's see the effect.

Of course, there are many bugs in the middle, but do you need to modify them slowly? you need to guard against them. do you need to adjust the bug yourself?

This is a simple arrangement. it may not be complete. However, there is a lot of intermediate design knowledge. you can't start talking about it. it is also a thought when you share it, some specific knowledge points can be used to find information.

The above is the detailed description of generating XML file instances using Shell scripts. For more information, see other related articles in the first PHP community!