Linux File and Content Search (recursive), grep, find

Content Search (recursion)

Grep/etc/httpd/modules/-LR '51ditu '*

File Name Search (recursive, not suitable for modules search)

Find/etc/httpd/-name httpd. conf

 

Reference:

GrepCommand is used to searchPatternThe mode specified by the parameter, and write each matching row to the standard output. These modes are qualified regular expressions that useEdOrEgrepCommand style.GrepCommand uses the uncertain compression algorithm.

IfFileMultiple Names are specified in the parameter,GrepThe command will display the name of the file that contains Matching lines. Characters with special meanings for shell ($, *, [, |, ^, (,),/) appear inPatternParameters must contain double quotation marks. IfPatternA parameter is not a simple string and must be enclosed in single quotes. For example[A-Z]In expressions such as-(minus sign), a range can be specified based on the sequence being sorted. Sorting order can define equivalent classes for use in character ranges. If no file is specified,GrepIt is assumed to be a standard input.

 

Note:

1. Do not run special filesGrepCommand to generate unpredictable results.
2. The input line should not contain null characters.
3. The input file should end with a line break.
4. Line breaks do not match regular expressions.
5. Although some labels can be specified at the same time, some of them will overwrite others. For example,-LThe option takes precedence over all other labels. In addition, if you specify-EAnd-FFlag.

Flag

 

-B	Add the block number that finds the row before each row. This flag helps you locate the disk block number through context.-BFlag cannot be used for input from standard input or MPs queue.
-C	Only the Count of matched rows is displayed.
-E	Treat each specified mode as an extended regular expression (ERE ). The Null Value of ere will match all rows.   Note:With-EFlagGrepCommands are equivalentEgrepCommand, but their errors and usage information are different and-SThe logo has different functions.
-EPatternlist	Specify one or more search modes. This is equivalent to a simple mode, but it is very useful when the mode starts with-(minus sign. The modes should be separated by line breaks. You can specify the null mode by using two linefeeds consecutively or adding a linefeed ("/n) after quotation marks. Unless both-EOr-FOtherwise, each pattern is considered as a basic regular expression (BRE ).GrepMultiple accepted-EAnd-FFlag. When a row is matched, all specified modes are used, but the evaluation sequence is not specified.
-F	Treat each specified pattern as a string rather than a regular expression. A null string can match all rows.   Note:With-FFlagGrepCommands are equivalentFgrepCommand, but their errors and usage information are different and-SThe logo has different functions.
-FPatternfile	Specifies the file that contains the search mode. The modes should be separated by line breaks. Empty rows will be considered as null. Each mode is considered as a basic regular expression (BRE), unless both-EOr-FFlag.
-H	Do not append the name of the file containing the row after matching the row. When multiple files are specified, file names are not allowed.
-H	If-ROr-RIf the symbolic link of the referenced file directory is specified on the command lineGrepSearch for the Directory files referenced by the symbolic link and all files under it in the file hierarchy.
-I	The case sensitivity of letters is ignored during comparison.

 

-L	Only names of files containing matching rows are listed (once. File names are separated by line breaks. If the standard input is found(Standard input).-LSame-CAnd-NWhen any combination of a flag is used together, its function is similar to using only-LFlag.
-L	If-ROr-RAnd the symbolic link that references the file type directory is specified on the command line or encountered during File hierarchy transferGrepSearch for the Directory files referenced by the symbolic link and all files under it in the file hierarchy. If both-HAnd-L, The recently specified option on the command line will take effect.
-N	Place the relevant row number in the file before each row. The starting line number of each file is 1. When processing each file, the row counter is reset.
-P[Separator]	Displays the entire section containing matched rows. The Section will followSeparatorThe paragraph delimiters specified by parameters are separated. These delimiters are in the same format as the search mode. Rows that contain paragraph delimiters are only used as delimiters and are not included in the output. The default paragraph Delimiter is a blank line.
-Q	Prohibit all write operations to the standard output, whether or not they are matched rows. If the input line is selected, the system exits in the zero state.-QSame-CAnd-L,-NWhen any combination of a flag is used together, its function is similar to using only-QFlag.
-R	Recursively searches for directories. By default, follow the link to the directory.
-R	Recursively searches for directories. By default, do not follow the link to the directory.
-S	Do not write error messages because the file does not exist or cannot be read. Other error messages are not forbidden.
-V	Displays all rows that do not match the specified mode.
-W	Execute word search.
-X	Displays rows that match the specified mode precisely without other characters.
-Y	The case sensitivity of characters is ignored during comparison.
Patternlist	Specify one or more modes to be used in the search. These modes will be considered as being used-EFlag.
File	Specifies the name of the file to be searched in the mode. If notFileVariable, which uses standard input

 

Introduction to the find and grep commands and Regular Expressions

Two more useful commands and Regular Expressions

Before learning new shell programming knowledge, let's take a look at two more useful commands. Although these two commands are not part of the shell, but it is often used in shell programming. next we will look at the regular expression.

Find command

Let's first look at the find command. this command is quite useful when we use it to find files, but it is difficult for new Linux users to use it. In a certain program, it is tested by the options provided by it, action type parameter, and the execution result of a parameter affects the following parameters.
Before going into these options and parameters, let's look at a very simple example. suppose there is a file wish on our machine. we need to run this operation as root, so that we can search the entire machine:

# Find/-name wish-print
/Usr/bin/wish
#

As we can think of, he will print the search results. It's very simple, isn't it?
However, it takes some time for him to run, because he will also search for disks on the Windows machine on the network. linux machines mount the File System of Large Windows machines. he will also find the location at the same time, although we know that the file we are looking for is located on the Linux machine.
This is the application of the first option. If we specify the-mount option, we can tell the find command not to search for the mounted directory.

# Find/-mount-name wish-print
/Usr/bin/wish
#

In this way, we can still search for this file, but this time we did not search for the mounted file system.

The complete syntax of the find command is as follows:

Find [path] [Options] [tests] [actions]

PATH is a very simple part: we can use absolute paths, such as/bin or relative paths, such... if we need to, we can also specify multiple paths, such as find/var/home.

The main options are as follows:
-Before viewing the directory itself, depth must first search for the contents in the directory.
-Follow follows the Symbolic Link
-Maxdepths N: searches for one directory to more than N layers
-Mount (or-xdev). Do not search for other file systems.

The following are some test options. we can specify a large number of tests for the find command, and each test returns true or false. when the find command works, it will examine the files found in order and perform the tests they defined on the files in order. if a test returns a false result, the find command stops the file being examined and continues the following action. the following table lists some of the most commonly used tests. You can view the manual page to obtain the list of possible extensions that we can use with the find command.

-Atime n files accessed n days ago
-Mtime n files modified N days ago
-Name pattern: in addition to the path, the file name that matches the specified type. To ensure that the specified type is passed to the find command rather than being assigned to the shell immediately, the specified type must be referenced in quotation marks.
-Newer otherfile requires a new file compared to the otherfile File
-Type C files, and here C can specify a type. the most common is that D represents a directory, and F refers to a common file. for other file types, we can view the manual page.
-File owned by the user username
We can also use operators to test the combination. Most of them have two formats: short format and long format.
! -Not Test Anti-DDoS
-A-And all tests must be true.
-O-or test if one of the values is true

We can use parentheses to forcibly change the order of tests and operators. because these have special significance for shell, we also need to use the backslash to reference them as a whole. in addition, if we specify the matching type for the file name, we must also reference it with quotation marks, so that we can avoid them being extended by shell, so that they can be passed directly to the find command. therefore, if we want to write such a test, to find a file that is closer to or starts with a range than the X file, we need to write it in the following format:

/(-Newer x-o-name "_ *"/)

Now we need to try to find the file with the latest modification date closer to while2 in the current directory. We can use the following command:

$ Find.-newer while2-print
.
./Elif3
./Words.txt
./Words2.txt
./_ Trap
$

The command we used above seems to be good, but we also searched for the current directory file, which is not what we want. We are only interested in regular files. so we can add another test-type F:

$ Find.-newer while2-type F-print
./Elif3
./Words.txt
./Words2.txt
./_ Trap
$

Working principle:

How do these commands work? We specify that the find command should be searched in the current directory (.), what we want to find is a file that is newer than while2 (-newer while2), and if the test has been passed, we also need to test whether the file is a regular file (-type-f ). finally, we use the previously used action-print to verify the file we found.

The file we want to search for is a new file than the while2 file, or it must start with an underscore, but it must also be a regular file. this example shows how to use a combination for testing:

$ Find./(-name "_ *"-or-newer while2/)-type F-print
./Elif3
./Words.txt
./Words2.txt
./_ Break
./_ If
./_ Set
./_ Shift
./_ Trap
./_ Unset
./_
$

At this time, we can see that this is not a very difficult thing, isn't it? We must escape parentheses so that they will not be protected by Shell and will be referenced in quotation marks * so that they can be passed directly to the find command.

Now that we can search for files reliably, let's take a look at some of the collaboration we can perform when searching for specified files. once again, we want to emphasize that what we listed here is only some of the most common options. We can view the manual page to get all the sets.

-Exec command: execute a command. This is the most common action.
-OK command is similar to-exec, except that it prompts the user to confirm the command before executing the command to be executed.
-Print: print the file name.
-Ls: Use the LS command to list the current file
The-exec and-OK commands are used as part of the parameter sub-sequence in the same line until a terminator/is encountered. for-exec and-OK, the string {} is a special type on Jane and will be replaced by the absolute path of the current file.

This explanation may not be easy to understand, but an example may be a good example.

The following is a simple example:

$ Find.-newer while2-type F-exec LS-l {}/;
-Rwxr-XR-x 1 Rick 275 Feb 8./elif3
-Rwxr-XR-x 1 Rick 336 Feb 8./words.txt
-Rwxr-XR-x 1 Rick 1274 Feb 8./words2.txt
-Rwxr-XR-x 1 Rick 504 Feb 8./_ trap
$

As we can see now, the find command is quite useful. to make good use of this command, you only need some simple exercises. however, this kind of exercise may have to be costly, so we should do some experiments on the find command.

Grep command

The second very useful command we will see is the grep command, which is an uncommon name and is short for general regular expression parser ). we use the find command to find the desired file in our system, but we need to use the grep command to find the specified string in the file. in fact, the most common practice is to pass grep to-Exec as a command when using the find command.

The grep command can contain options, matching modes, and files to be searched:
Grep [Options] pattern [files]
If no file name is specified, the system searches for standard input.
Let's start with the main options of the grep command. Here we list only some of the main options. We can get a more detailed description from the manual.

-C: print the total number of matched rows instead of matching rows.
-E Open extension expression
-H do not use the file name in which the Matching content is found as the prefix of the output line.
-I case-insensitive
-L list the file names with matched rows instead of the actual matched rows.
-V converts the matching type to select unmatched rows instead of matched rows.
Some examples are as follows:
$ Grep in words.txt
When shall we three meet again. In thunder, lightning, or in rain?
I come, graymalkin!
$ Grep-C in words.txt words2.txt
Words.txt: 2
Words2.txt: 14
$ Grep-C-V in words.txt words2.txt
Words.txt: 9
Words2.txt: 16
$

Working principle:

In the first example, the grep command is simply used to find the character string in the words.txt file and print the matched rows. The file name is not printed here, because here we only use a file.
In the second example, print the total number of matched rows in two different rows. In this case, print the file name.
In the last example, we use the-V option to convert the search conditions and print the total number of unmatched rows in the two files.

Regular Expression

As we can see, the basic usage of the grep command is relatively easy to grasp. now let's take a look at the basic regular expressions, which will allow us to perform more complex matching. as we mentioned earlier, regular expressions are used in Linux or some other open source languages. we can use it in VI or when writing Perl scripts.

When using a regular expression, some characters are processed in different ways. The most common usage is as follows:

^ At the beginning of a row
$ At the end of a row
. Any single character
[] Square brackets contain the letter range, and any one of them can be matched, for example, the letter range of A-E, or we can use ^ for antsense.
If we want to use them as common characters, we need to add/before these characters. So if we want to find a $ character, we need to use/$ for searching.

The following are some useful special matches that can be used in square brackets:

[: Alnum:] alphanumeric characters
[: Alpha:] letters
[: ASCII:] ASCII characters
[: Blank:] space or tab
[: Cntrl:] ASCII control character
[: Digit:] Number
[: Graph:] non-Control, non-space characters
[: Lower:] lowercase letters
[: Print:] printable characters
[: Punct:] punctuation
[: Space:] blank characters, including vertical tabs
[: Upper:] uppercase characters
[: Xdigit:] hexadecimal number

In addition, if the-e option is used to specify the extended matching, the character after the regular expression may be a combination of some other control matching types. if we only want to use them as common characters, we also need to add the Escape Character /.

? Optional, but only once
* 0 or multiple projects must be matched.
+ One or more projects must be matched.
{N} must match n times
{N,} must match n times or more times
The matching range of {n, m} is n to m, including M.

These contents seem to be complicated, but if we step by step, we will find that the content is not as complicated as we will first see. the simplest way to grasp regular expressions is to give a few examples:

If we want to find the line ending with character E, we can use the following command:

$ Grep e $ words2.txt
Art thou not, fatal vision, sensible
I see thee yet, in Form as palpable
Nature seems dead, And wicked dreams abuse
$

As we can see, this command will search for Matching lines ending with E.
Now let's look for words ending with the letter. to achieve this goal, we use special matching in square brackets. in this case, we use [[: blank:] to test a space or a tab:

$ Grep A [[: blank:] words2.txt
Is this a dagger which I see before me,
A dagger of the mind, a false creation,
Moves like a ghost. Thou sure and firm-set earth,
$

Now let's look for a word with three letters starting with th. in this case, we need to use [[: Space:] at the same time to determine the end of a word and use it. to match another letter:

$ Grep th. [[: Space:] words2.txt
The handle toward my hand? Come, let me clutch thee.
The curtain 'd sleep; Witchcraft celebrates
Thy very stones prate of my whereabout,
$

Finally, we need to use the extended grep command to find words with 10 characters long and lowercase letters. here we need to specify a character range to match A to Z, and specify 10 duplicates of the character:

$ Grep-E [A-Z]/{10/} words2.txt
Proceeding from the heat-oppressed brain?
And such an instrument I was to use.
The curtain 'd sleep; Witchcraft celebrates
Thy very stones prate of my whereabout,
$

Here we are just a more important part of getting involved with regular expressions. as most of the other content in Linux, there will be a lot of documents to help us find more detailed content, however, the best way to learn regular expressions is to experiment with these expressions.

Command Execution:

When writing a script, we often need to obtain the command execution result in the shell script. that is to say, we need to execute a command and put the output result of this command in a variable. in this case, we can use the $ (command) syntax described in the previous set command example. this is also a relatively old format, and the most common use is the 'command' format.

All new scripts should use $ (...) format, which can be used to avoid some complicated conversion rules caused by the use of $, ',/in the anti-quotes command. if '... 'The structure uses backticks, so we need to use/for escape. these relatively fuzzy characters may confuse the program, and sometimes even some experienced programs have to perform some experiments so that the quotation marks in the reverse quotation mark command can work correctly.

The result of the $ (command) command is only the output of a simple command. Here we should note that this is not the return status of this command, but the output string. the following example:

#! /Bin/sh
Echo the current directory is $ pwd
Echo the current users are $ (WHO)
Exit 0

Because the current directory is a shell environment variable, the first line does not need to use this command execution structure. however, if you want the result of the WHO command to be visible in this script, we need to use this command structure.

If we want to put their results in a variable, we can assign them to a variable as usual:

Whoisthere = $ (WHO)
Echo $ whoisthere

The ability to put the execution result of a command in a script variable is quite powerful, because it is easy to use the current command in the script and get their output. if you find that you are trying to convert the parameter set of a standard command output result on the standard output and use them as a program parameter, you will find that the xargs command will help you complete all this. you can view the manual page for more details.
Sometimes a problem occurs when the command we want to call outputs some blank characters or more content than we want before the expected text appears. in this case, we can use the set command we mentioned earlier.

Arithmetic Extension

We have used the expr command, which can handle simple arithmetic commands, but its execution is quite slow, because a new shell needs to be called when processing the expr command.

A better replacement is $ ((...)) extension. wrap the expected expression in parentheses so that $ ((...)) and we can perform more effective simple arithmetic operations.

Example:

#! /Bin/sh
X = 0
While ["$ X"-ne 10]; do
Echo $ x
X = $ ($ x + 1 ))
Done
Exit 0

Parameter Extension

We have already seen the simplest form of parameter allocation and extension, where we write:

Foo = Fred
Echo $ foo

A problem occurs when we add another character at the end of a variable. suppose we want to write a short script to process files named mongotmp and 2_tmp. We can try to use the following script to process them:

#! /Bin/sh
For I in 1 2
Do
My_secret_process $ I _tmp
Done

However, in each loop, we will get the following information:

My_secret_process: Too few arguments

What errors have occurred?

The problem is that shell will try to replace the variable $ I _tmp with its variable value, but this variable does not exist. shell does not regard this as an error, but replaces it with a null value. Therefore, no parameter is passed to my_secret_process. to protect $ I extension as part of a variable, we need to put I in a pair of curly brackets:

#! /Bin/sh
For I in 1 2
Do
My_secret_process $ {I} _ TMP
Done

In this way, in the first loop, the I value will be replaced with $ {I} to give an actual file name. in this way, we have replaced the value of a parameter with a string.

We can do a lot of replacement in shell. This method often provides an elegant solution to the parameter processing problem.

Common tables are as follows:

$ {Parm:-default} If a parameter is null, it is set to a default value.
$ {# Parm} indicates the length of the parameter.
Starting from the end of $ {parm % word}, remove the smallest part that matches the word and return the remaining part.
Starting from the end, $ {parm % word} removes the longest part that matches the word and returns the rest.
Starting from $ {parm # Word}, remove the smallest part that matches the word and return the remaining part.
Starting from $ {parm # Word}, remove the longest part that matches the word and return the remaining part.

These replicas are quite useful for processing strings. the last four parts can be used to remove part of the content in the string, which is more useful for processing file names and paths. as shown in the following examples:

#! /Bin/sh
Unset foo
Echo $ {FOO:-bar}
Foo = fud
Echo $ {FOO:-bar}
Foo =/usr/bin/X11/startx
Echo $ {Foo #*/}
Echo $ {Foo ##*/}
Bar =/usr/local/etc/local/networks
Echo $ {bar % local *}
Echo $ {bar % local *}
Exit 0

If we run this script, we will get the following output:

Bar
Fud
Usr/bin/X11/startx
Startx
/Usr/local/etc
/Usr

Working principle:

In the first sentence, $ {FOO:-bar} is specified as bar for the foo value, because no value is specified for Foo when the statement is executed. the value of Foo remains unchanged until it encounters an unset statement.

Here we have some things to note:

$ {FOO: = bar} will set the variable $ Foo. this string operator detects that Foo exists and is not null. if it is not empty, its value is returned, but in the opposite case, the foo value is set to bar and the replaced result value is returned.
$ {FOO :? Bar} prints out FOO: bar. If Foo does not exist or is set as a null value, it exits the command.
Finally, $ {FOO: + bar}. If Foo exists and is not empty, bar is returned.
The {Foo # */} statement matches and only removes the content on the left (here we need to remember that * matches 0 or multiple characters ). {Foo # */} matches and removes as many content as possible, so it removes the rightmost/and all the characters above it.
The {bar % local *} statement matches the local character from the right until the first occurrence, and the {bar % local *} matches as many characters as possible from the right, until the first local.

Because UNIX and Linux depend heavily on the concept of filtering, we often need to manually redirect the execution results of an operation. suppose we want to use the cjpeg command to convert a GIF file to a jpeg file:

$ Cjpeg image.gif> image.jpg

Sometimes we may perform such operations on a large number of files. How can we automatically redirect them? We can easily do this:

#! /Bin/sh
For Image in *. gif
Do
Cjpeg $ image >$ {image % GIF} JPG
Done

This script can convert each GIF file in the current directory into a jpeg file.