The "Swiss Army Knife" in the Linux command line

Source: Internet
Author: User
Tags sort

The "Swiss Army Knife" here refers to the work of a simple command that can complete a large chunk of code in other high-level languages.

The following content is a summary of the Joshua Levy on Quora's website:

Get the intersection, collection, and difference of the contents of the file through Sort/uniq: Suppose there are two text files in a and B, the file itself has been removed from the duplicate content. Here is the most efficient way to handle any volume of files, even several G files. (Sort does not require memory, but you may need to use the-t argument.) You can try to compare, and you can see how many lines of code would be needed if you were to use Java to handle merging files on disk.

Copy Code code as follows:


Cat A B | Sort | Uniq > C # C is the collection of A and B


Cat A B | Sort | uniq-d > C # C is the intersection of a and B


Cat A b b | Sort | Uniq-u > C # c is the difference between A and B



sum up the sum of the third column of a text content (this method is 3 times times faster than in Python and requires only 1/3 of the amount of code):

Copy Code code as follows: awk ' {x + = $} end {print x} ' myfile



If you want to see the volume and date of a file in a directory tree, use the following method, which is equivalent to "Ls-l" in your directory, and the output is more readable than you would use "LS-LR":

Copy Code code as follows:


find. -type F-ls


Use the Xargs command. This command is very powerful. Notice how many things you can control on each line. If you're not sure it's the correct execution, use Xargs echo first. Similarly,-i{} is also very useful. Example:

Copy Code code as follows:


find. -name *.py | Xargs grep some_function


Cat Hosts | xargs-i{} SSH root@{} hostname



Suppose you have a text file, such as a Web server log, with some values on some lines, such as the acct_id parameter in the URL. If you want to count all request records for each acct_id:

Copy Code code as follows:


Cat Access.log | Egrep-o ' acct_id=[0-9]+ ' | cut-d=-f2 | Sort | uniq-c | SORT-RN]

Related Article

E-Commerce Solutions

Leverage the same tools powering the Alibaba Ecosystem

Learn more >

Apsara Conference 2019

The Rise of Data Intelligence, September 25th - 27th, Hangzhou, China

Learn more >

Alibaba Cloud Free Trial

Learn and experience the power of Alibaba Cloud with a free trial worth $300-1200 USD

Learn more >

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.