Efficient command line skills for Linux Users

I recently saw a Q & A question on Quora about saving time Tips for Linux users in high efficiency. Summarize the answers to this question and add some of your own experiences. record them as follows for your reference. The following describes some command line tools that are useful in the daily work of several respondents.

For any commands you do not know, use "man <COMMANDNAME>" to view them, or use Google.

Some commands must be installed with the yum, apt-get install command.

I. Basic commands

1. Understand the basic bash: Read the entire bash man page.

2. Learn VIM: on Linux, although you have Emacs and Eclipse, VIM is still a powerful tool.

3. Understand SSH and basic password-free authentication methods. For example, use ssh-agent or ssh-add. Lingxi zhiqui usually uses the following script to complete password-free verification, saving time and effort.

Execution method sh nopasswd USER REMOTE_HOST

Before executing this script, confirm:

• Id_dsa.pub already exists on the local machine. If no. Run the ssh-keygen-t dsa command to obtain the result.
• Log on to the user's home directory on the remote machine. The. ssh folder already exists. If no. ssh folder is created.

 
 

  
  
$ cat  nopasswd 
  
  
#!/bin/sh 
  
  
scp ~/.ssh/id_dsa.pub  $1@$2:~/ 
  
  
ssh $1@$2 " touch ~/.ssh/authorized_keys ; cat ~/id_dsa.pub  >> ~/.ssh/authorized_keys; chmod 644 ~/.ssh/authorized_keys; exit" 
 
 

4. Familiar with common Bash task management commands: &, Ctrl-Z, Ctrl-C, jobs, fg, bg, and kill.

5. Basic file management commands: ls, ls-l, less, head, tail, tail-f, ln, ln-s, chmod, chown, du, du-sk *, df, mount

6. Basic network management commands: ipconfig, ifconfig, and dig

7. Familiar with regular expressions and the options used by grep and egrep:-o,-A,-B

8. Software Installation commands: apt-get and yum

Cat-n: displays the row number.

Ii. Some Expressions

!! : Execute the previous command again.

! $: Last word of the previous command

{A. B}: a Number list in the order from a to B

{A, B, c}: three words a, B, c. You can use touch/tmp/{foo, bar, baz} like this}

{$1-$9}: command line parameters for executing shell scripts

$0: name of the command being executed

$ #: Number of parameters input in the currently started command

$? : The execution return value of the previous command.

$: Process ID of the shell.

$ *: Start all parameters of the shell script from $1.

Iii. Routine commands

Ctrl-R: In bash, Ctrl-R is used to search for historical commands.

Ctrl-W, Ctrl-U, Alt-BackSpace: bash, Ctrl-W Delete the last word, Ctrl-U Delete the last line, alt-BackSpace Delete the word man readline before the cursor contains a large number of default hotkey bindings in bash.

Cd-: returns the previous working path.

Xargs: Very powerful command. If you are not sure whether the task can be correctly executed, you can check it with xargs echo. The following is an example of using this function:

 
 

  
  
find . -name \*.py | xargs grep some_function 
  
  
cat hosts | xargs -l {} ssh root@{} hostname 
 
 

Parallel: a more powerful command. It can execute tasks in parallel, separate input files, and specify multiple nodes to run commands at the same time. For more information, see this link.

Pstree-p: a powerful tool for using the process tree

Pgrep, pkill: use the name to find the process, or directly send a signal to the process with the specified name.

Understand the signal that a user can send to a process. For example, kill-STOP [pid] to suspend the pid process.

Nohup, disown, screen, and tmux: These two commands are useful when you need to keep processes running in the background forever.

Lsof, netstat-lntp: queries the current process listening port.

Set: In the bash script, use set-x to get the debug output and set-e to get the error output.

;: A semicolon is used to open a sub-shell and close it after running. For example:

 
 

  
  
# Execute some commands in the current path
  
  
(Cd/some/other/dir; other-command)
  
  
# The working path is still the current directory
 
 

Learn about multiple parameter expressions in shell: $ {name :? Error message}

Check whether a variable exists. If no error message exists.

$ {Var % suffix}, $ {var # prefix}: Output part of the var variable except the prefix or suffix. For example, the output code is foo.txt.

 
 

  
  
var = foo.pdf 
  
  
echo ${var%pdf}.txt 
 
 

<,>: Input/Output redirection.

Some_command> logfile 2> & 1: output the standard output and standard error output during the running of some_command to the logfile.

Man ascii: Get an easy-to-use ASCII table that contains 10-digit and 16-digit values.

Screen and dtach: in remote ssh painting, use these two commands to save your session and avoid interruption due to network problems.

Curl, curl-l, wget: in web page debugging, these commands can help you download web Page code, which is very useful.

Lynx-dump-stdin: convert HTML to text

Xmlstarlet: this command is useful when you need to process XML.

Ssh-L, ssh-D: This command helps you establish an ssh tunnel between the remote server and your machine when you need to access the webpage using a remote server.

Ssh connection optimization: The following configuration can help you avoid link loss. You do not need to enter yes for confirmation every time and enable compression on the link. We recommend that you put it in. ssh/config.

 
 

  
  
TCPKeepAlive=yes 
  
  
ServerAliveInterval=15 
  
  
ServerAliveCountMax=6 
  
  
StrictHostKeyChecking=no 
  
  
Compression=yes 
  
  
ForwardAgent=yes 
 
 

Add # Before the command that is being entered. If you have already entered half of the command, you can use Alt-# To Add '#' before the command '#', change the entire command into a comment. In this way, you can find the command later in the command history.

Cron: helps you develop scheduled tasks.

Ctrl-S Ctrl-C: When you accidentally need to output a large amount of text, enter these two operations in sequence, which is faster than simply pressing Ctrl-C to terminate the program.

Iv. Data Processing

Sort, uniq, uniq-u, uniq-d: understand these sorting commands.

Cut, paste, and join: Understand the maintenance tools for these text files. Many people forget to join after using cut.

Use sort/uniq to perform the intersection, addition, and complement operations of the set =

Assume that a and B are two text files, and the rows are unique.

The following commands can quickly implement some set operations.

 
 

  
  
cat a b | sort | uniq > c   # c is a union b 
  
  
cat a b | sort | uniq -d > c   # c is a intersect b 
  
  
cat a b b | sort | uniq -u > c   # c is set difference a - b 
 
 

Using LC_ALL = C: Linux locale settings will affect a large number of command line tools, including sorting tools. In most installed Linux systems, LANG or other locale are set to US English by default. However, this may lead to several times slower sorting and other commands. Therefore, export LCALL = C can avoid using i18n to process data and improve performance.

Awk, sed: These two tools can implement complex data replacement and modification.

For example, the following command calculates the sum of the data in the Middle and Lower columns of a text file.

Using shell to perform this operation is three times faster than using Python.

 
 

  
  
awk '{ x += $3 } END { print x }' 
 
 

Shuf: This command can be used to mix the lines in a file or randomly select some lines.

Sort: Understand how common sort options (-t,-k,-s) work. Note:-k1 and 1 will only sort the first column, while-k1 will sort the entire row. -S can achieve stable sorting.

For example, first sort by the second domain and then by the domain, you can use this command:

Cat INPUT_FILE | sort-k1, 1 | sort-s-k2, 2 tab input );

To enter a tab in the bash command line, use Ctrl-V <tab> or $ '\ t.

Hd and bvi: For binary files, these two commands perform hexadecimal extraction and binary editing respectively.

Strings, grep: Helps search for text in binary files.

Iconv, uconv: Helps convert text encoding

Split and csplit: Separate files by size and by specific mode.

V. system debugging

Iostat, netstat, top, atop, htop, and dstat: helps you understand the hard disk, CPU, memory, and network status. This helps you first understand what is happening in the system.

Free, vmstat: These two commands are important if you want to know the memory status. Cached indicates the File Cache size in the Linux kernel.

Kill-3 <pid>: When debugging a Java program, you can use this command to find the complete stack trace and heap information (including garbage collection details) in stderr/logs ).

Mtr and traceroute: helps you find network problems. The former is better than traceroute.

Iftop and nethogs: These two commands can be used to find out which port or how much network bandwidth the process occupies.

AB, siege: This Apache tool can help you quickly check the performance of web servers.

Wireshark and tshark are powerful tools for more advanced network debugging.

Strace and ltrace: These two commands can help you provide some clues about program failures, false positives, crashes, and other issues without knowing anything about them.

In addition, they can help find some performance problems. For example, the-c option can be used for profiling, and the-p option can be mounted to a specified process.

Ldd: Check the Shared Library

Gdb: Learn How to Use GDB to connect to a running process and obtain its stack trace.

/Proc/: useful for on-site debugging. For example,/proc/cpuinfo,/proc/XXX/cwd,/proc/XXX/exe,/proc/XXX/fd/,/proc/XXX/smaps

Sar: When you need to determine why a system error occurred at a certain time, this command displays the CPU, memory, and network history.

Stap and perf: These two tools are useful when a deeper analysis system and performance are required.

Dmesg: This is useful when the system encounters abnormal phenomena, such as hardware or driver problems.

Vi. References

Http://unixhelp.ed.ac.uk/scrpt/scrpt2.2.2.html

Http://www.quora.com/Linux/What-are-some-time-saving-tips-that-every-Linux-user-should-know

VII. Thank you

• Yang Qing: Increase Alt-BackSpace
• Dikar: Modify string-> strings
• Zsc: added the GNU parallel command.
• RobberPhex: Modify the record-> abnormal