Linux 15th week

1. Summarize the detailed usage of SED and awk;

Sedsed is essentially an editor, but it is non-interactive, and it is a character stream oriented, and the input character stream is output after sed processing. SED itself is a pipeline command, can be analyzed standard input, mainly used to analyze the use of keywords, statistics, etc., in addition to the data can be replaced, deleted, select specific lines and other functional formats: SED&NBSP;[OPTION]&NBSP;   ' script '  inputfile...script: ' Address command ' common option:-N: Do not output the contents of the mode to the screen, only list the line that has been processed by SED. -e: Multi-point editing, which performs SED motion editing directly in command line mode. -F: Writes the action of SED in a file,-f filename can perform the sed  action within the filename. -R: Supports the use of extended regular expressions;-I: Directly modifies the contents of the read file instead of the screen output. Address delimitation: (1) do not give address: the full text of the processing; (2) Single address: #: The specified line;/pattern/: Each row that the pattern can match to; (3) address range: #,##,+#/part1/,/part2/ Edit command: D: delete P: Content in display mode space a \text: Append text after line, support to use \ n to implement multiline append, I \text: Insert text before line, support to use \ n to implement multiline insertion; c \ Text: Replace the line of behavior or multiline text; W /path/to/somefile: Saves the pattern space to the specified file; R /path/from/some: Reads the text of the specified file to the line in the pattern space that matches the line; : Prints line numbers for lines in pattern space;!: Reverse condition; s///: Supports use of other delimiter,[email protected]@@,s###; substitution tags: g: inline global substitution; p: Displays the row where the substitution succeeded; w /path/to/ Somefile: Saves the result of the replacement successfully to the specified file; Frequently used cases with new additions to the Behavior Unit 1, nl /etc/passwd | sed  ' 2,5d ' Description: sed action for ' 2,5d ', D is the deletion, the result of the command run is to delete the 2~5 line. The actions that follow the SED must be enclosed in ' two single quotes. If you want to delete only the second line, the command is nl /etc/passwd | sed  ' 2d ' If you want to removeIn addition to line 2nd to the last line, you can write:nl /etc/passwd | sed  ' 2, $d ' 2, nl /etc/passwd | sed  ' 2a  play game ' Description: The command execution effect is after the second line (that is, the third line) with the word "play game" if you want to precede the second line with a character device, you can: NL&NBSP;/ETC/PASSWD  | sed  ' 2i play game ' NOTE: the addition of a in 2a means that the second line is followed, while the 2i I is the meaning of the insertion, referring to the front of the second line. 3, nl /etc/passwd | sed  ' 2a play game or ...\nplay football ' Note: The result of the command execution is to add 2 lines after the second line. The addition of new rows must be followed by \ n after each line to replace and display the behavior Units 1, nl /etc/passwd | sed  ' 2,5c no 2~5 number ' Description: The execution effect of the command is to replace the contents of line 2nd to 5th with "No 2~5 number" 2, nl /etc/passwd | sed -n  ' 5,7p ' Description: The execution effect of the command is to print out only the 5th to 7th line in the file in the command-N to represent the quiet mode! 3, sed  ' s/to be replaced content/new content/g ' Description: The effect of command execution is to replace the specified content 4, record the process of obtaining an IP address first step: First query Ip/sbin/ifconfig eth0 Note: The purpose is to obtain IP data, So first use the keyword to find that line. Step two: Use the keyword configuration grep to select a key row of data/sbin/ifconfig eth0 | grep  ' inet addr ' Note: Because only IP data is required, the next step is to delete the unwanted content, and a regular expression is required to help implement: ^.*inet addr The first part of the IP is deleted/sbin/ifconfig eth0 | grep  ' inet addr '  | sed  ' s/^.*inet addr://g ' NOTE: The above command is to delete the data in front of the IP, The next step is to delete the data behind the IP, at which point the regular expression is: bcast.*$ Fourth: Remove the part after the IP/sbin/ifconfig eth0 | grep  ' inet  Addr '  | sed  ' s/^.*inet addr://g '  | sed  ' s/bcast.*$//g ' This intercepts the IP address. Directly modify the contents of the file (caution) 1, sed -i  ' $a   #This  is a test '   Test.txt Description: The execution result of the command is to add "this is a test" to the last line of Test.txt because $ represents the last line, and A's operation is new, so the file is finally added. awk feature: awk is a data processing tool. While SED is often used for a full line of processing, awk prefers to divide a line into several "fields" to handle. As a result, awk is well suited for handling small data processing. Basic syntax:awk [options]  ' program '  file ...program:pattern {action statements} Multiple statements are separated by semicolons with the print,printf option:-F: Indicates the field delimiter used in the input;-v var=value: custom variable; 1, printprint item1, item2,  ... Key points: (1) comma delimiter, (2) The output of each item can be a string, or a numeric value, the current record of the field, variable or awk expression, (3) as omitted item, equivalent to PRINT&NBSP;$0;2, variable (1) built-in variables fs:input  Field seperator, enter delimiter, default is white space character, ofs:output field seperator, output delimiter, default is white space character; Rs:input record&nbsP;sepeator, line feed at input, ors:output record seperator, line break at output, Nf:number of field, number of fields {print  nf}, {print  $NF}nr:number of record, number of rows; FNR: Each file counts; number of rows; FileName: current file name; ARGC: Number of command-line arguments; ARGV: Array , the parameters given by the command line are saved, and (2) The custom variable-v var=value  variable name distinguishes the character case in the program direct definition 3, the printf command formatted output: Printf format, &NBSP;ITEM1,&NBSP;ITEM2,&NBSP, ..... (1) format must be given, (2) does not wrap, need to explicitly give a newline control,\n  (3) format in the need to specify a formatting symbol for each subsequent item; Format:%c: ASCII code for displaying characters;%d,%i: displaying decimal integers ;%e,%e: Numerical display of scientific notation;%f: Displayed as floating-point number;%g,%g: Displays values in scientific or floating-point form;%s: display string;%u: unsigned integer; percent: show% itself; modifier: #[.#]: The width of the first digital control display; Represents the precision after the decimal point;%3.1f-: Left justified +: Displays the value of the symbol 4, pattern (1) Empty: null mode, matching each row, (2)/regular expression/: Processing only rows that can be matched to this pattern ; (3) Relational expression: relational expression; The result is "true" has "false"; The result is true: The result is a non-0 value, a non-empty string, and (4) Line ranges: Row range, startline, ENDLINE:/PAT1/,/PAT2/Note: The format of the direct given number is not supported (5) begin/end mode begin{}: Executes only once before the text in the file is started, end{}: Only once the text processing is complete; 5, The commonly used action (1) Expressions (2) control statements:if,while etc; (3) Compound statements: Combined statement; (4) input  Statements  (5) OUTPUT&NBSP;STATEMENTS6, control statement (1) If-else syntax: if (condition)  statement [else statement] Usage scenario: Make a conditional judgment on the entire row or field obtained by awk; example:awk -f:  ' {if ($3>=500)  {printf  ' common user:%s\n ', \ n}  else {printf  "root or sysuser:%s\n", "$ {}} '  /etc/passwdawk -F:  ' {if ($NF = = "/bin/bash")  print $1} '  /etc/passwdawk  ' {if (nf>5)  print $0} '  /etc/ fstabdf -h | awk -f%  '/^\/dev/{print $1} '  | awk  ' {if ($NF >=20)  print $1} ' (2) while loop syntax: while (condition)  statement condition   "true", loop with condition "false", exit loop ; Usage scenario: used when multiple fields within a row are processed one at a time, and each element in an array is processed in a single process; example:awk  '/^[[:space:]]*linux16/{i=1;while (I&LT;=NF)  {print   $i, Length ($i);  i++}} '  /etc/grub2.cfgawk  '/^[[:space:]]*linux16/{i=1;while (I&LT;=NF)   {if (length ($i) >=7) {print  $i, Length ($i)}; i++}} '  /etc/grub2.cfg (3) Do-while loop syntax: do statement while (condition) meaning: perform at least one loop body (4) For loop languageMethod: for (EXPR1;EXPR2;EXPR3)  statementfor (variable assignment;condition;iteration process) {For-body } Example:awk  '/^[[:space:]]*linux16/{for (i=1;i<=nf;i++) {print  $i, Length ($i)}} '  /etc/grub2.cfg   Special usage: Ability to iterate through elements in an array; syntax: for (Var in array) {for-body} (5) switch statement syntax: switch (expression)  {case  VALUE1 or /REGEXP/:statement;case VALUE2 or /REGEXP2/:statement;...; Default:statement} (6) Next closes the processing of the bank in advance and goes directly to the next line; example:awk -f:  ' {if ($3%2!=0) next;print $1,$3} '  / Etc/passwd7, array associative arrays:array[index-expression]index-expression:  (1) can use any string, string to use double quotation marks, (2) If an array element does not exist beforehand, at the time of reference , awk automatically creates this element and initializes its value to "empty string"; To determine if an element exists in the array, use the "Index in array" format, and weekdays[mon]= "Monday" to iterate over each element in the array. To use a For loop; for (Var in array) {For-body} example:awk  ' begin{weekdays["Mon"]= "Monday" weekdays["Tue"]= " Tuesday "; for (i in weekdays) {Print weekdays[i]}} ' 8, other common examples (1) count the number of occurrences of each file system type in the/etc/fstab file; awk   '/^uuid/{fs[$3]++}end{for (i iN&NBSP;FS) {Print i,fs[i]}} '  /etc/fstab (2) counts the number of occurrences of each word in the/etc/fstab file; awk  ' {for (i=1;i<=nf; i++) {count[$i]++}}end{for (i in count) {Print i,count[i]}} '  /etc/fstab

2. Delete whitespace characters from the beginning of all lines in the/boot/grub/grub.conf file;

[[Email protected] ~]# sed ' s/^[[:space:]]*//g '/boot/grub/grub.conf

3. Delete all # and white space characters at the beginning of the line at the beginning of #, followed by at least one white-space character, in the/etc/fstab file;

[[Email protected] ~]# sed ' s/^#[[:space:]]\+//'/etc/fstab

4. Save the odd line of/etc/fstab file as/tmp/fstab.3;

[[Email protected] ~]# sed ' n;d '/etc/fstab >/etc/fstab.3

5. Echo a file path to the SED command, take out its base name, and further, take out its path name;

[Email protected] ~]# echo "/etc/sysconfig/network/" | Sed-r ' [Email protected]^/.*/([^/]+)/[email protected]\[email protected] ' [[email protected] ~]# echo '/etc/sysconfig/ network/"| Sed ' s#[^/]\+/\?$## '

6. Count the occurrences of each word in all lines in the specified file;

[[email protected] ~]# awk ' {for (i=1;i<=nf;i++) {count[$i]++}}end{for (i in count) {print I,count[i]}} '/etc/fstab

7. Count the number of States of all TCP connections on the current system;

[Email protected] ~]# Netstat-tan | awk ' Fnr>2{print $NF} ' | Sort | Uniq-c

8. Count the number of resource accesses for each IP in the specified Web Access log:

[[email protected] ~]# awk ' {ip[$1]++}end{for (i in IP) {print i,ip[i]}} '/var/log/httpd/access_log

9. Write a script: Define an array whose elements are the names of all files that end with. Log in the/var/log directory; Displays the number of rows per file;

[Email protected] ~]# vim line.sh#!/bin/bashtest= ' ls/var/log/*log ' for I in $testdo wc-l $idone

10, write a script, can randomly pick a classmate from all the students to answer questions ; further: A parameter can be accepted as the number of classmates to be selected;

[[[email protected] ~]# vim student.sh#!/bin/bashstu = (Zhangsan lisi wangwu zhaoliu liuhu zhaosi) i=$[$RANDOM  % ${#stu [@]}]echo  ${stu[i]}[[email protected] ~]# vim students.sh#!/bin/bashstu= (Zhangsan lisi  wangwu zhaoliu liuhu zhaosi) read -p  "Please input the number  of students: " numif [[  $num  -le ${#stu [@]} ]]then     for  ((i=0;i<num;i++))     do        x =$[$RANDOM  % ${#stu [@]}]        echo ${stu[$x]}         stu[$x]=${stu[${#stu [@]}-1]}         unset stu[${#stu [@]}-1]    doneelse    echo  " input error! " Fi 

11, authorized CentOS users can run the FDISK command to complete Disk Management, and use MKFS or MKE2FS to achieve file system management;

[Email protected] ~]# Vim/etc/sudoerscentos all= (Root) nopasswd:/sbin/fdisk,/SBIN/MKE2FS,/SBIN/MKFS

12, authorized Gentoo users can run the logical volume management of the relevant commands;

[[email protected] ~]# Vim/etc/sudoersgentoo all= (Root) LVM

13, based on the pam_time.so module, restrict the user through the SSHD service remote login only during working hours;

[Email protected] ~]# Vim/etc/pam.d/sshdaccount required pam_time.so[[email protected] ~]# Vim/etc/security/time.con f *;*;*; motuwethfr0900-1800# means working hours from 9 o'clock to 6 o'clock in the afternoon allow access to SSH

14, based on the Pam_listfile.so module, define only some users, or some groups of users can log in the system; "

[Email protected] ~]# vim/etc/sshd_userlistrootcentosgentoo[[email protected] ~]# chmod 600/etc/sshd_userlist [[email] Protected] ~]# chown root/etc/sshd_userlist [[email protected] ~]# vim/etc/pam.d/sshd add auth required Pam_listfile . So Item=user sense=allow file=/etc/sshd_userlist onerr=succeed

This article is from the "Chase Dream" blog, please be sure to keep this source http://sihua.blog.51cto.com/377227/1877945

Linux 15th week