Proficient in regular expressions (metacharacters)

In early January, I gave a lecture on regular expressions at the company. Here I hope to share some of the things that I have worked hard for that month, first, I would like to share a Familiar Book "proficient in regular expressions", which is indeed a good book. But I wouldn't have read it twice in a month if it wasn't for work reasons! It's really boring! However, it has been found to be helpful to you recently. It can also be frequently used at work, and sometimes it can help you solve some problems in the group. I have simplified the version of this book! I did not have the details on the book, but I combined some online materials with my own experiments.

　　1. Historical Origins　　

The "originator" of regular expressions may be traced back to early research by scientists on the working principles of human nervous systems. Warren McCulloch from New Jersey and Walter Pitts, born in Detroit, developed a new mathematical method to describe neural networks, they innovatively describe neurons in the nervous system as small and simple automatic control elements, thus making a great work innovation.

In 1956, a math scientist named Stephen Kleene in the United States, based on Warren McCulloch and Walter Pitts's early work, I published a paper titled neural network event representation. I used mathematical symbols called regular expressions to describe this model and introduced the concept of regular expressions. A regular expression is used as an expression to describe it as an "Algebra of a regular set". Therefore, the term "Regular Expression" is used.

Later, people found that the work could be applied to other aspects. Ken Thompson applied this result to some early research on computational search algorithms. Ken Thompson is the principal inventor of UNIX, the father of the famous UNIX. The father of UNIX introduced the symbol system into the editor QED, followed by the editor ed on UNIX, and finally introduced grep.

UNIX grep families include grep, egrep, and fgrep. The commands of egrep and fgrep are only slightly different from those of grep. Egrep is an extension of grep and supports more metacharacters. fgrep regards all letters as words, that is, the metacharacters in regular expressions indicate their literal meanings, not special.

Many UNIX tools support regular expressions. In the past two decades, in the Windows camp, the idea and application of regular expressions have been supported and embedded in most Windows developer kits! Currently, among the mainstream development languages (PHP, C #, Java, C ++, VB, JavaScript, Ruby, and Python) and hundreds of millions of applications, you can see the graceful dance of the regular expression.

2And egrep metacharacters　　

One of the simplest applications of Text Retrieval regular expressions-many text editors and word processing software provide the regular expression retrieval function. Egrep is the simplest and most popular one. Almost all genres (here I call different languages or processing software a genre) Support metacharacter extensions or modifications based on egrep.

　　2.1egrep metacharacters ^ $

The delimiters '^' and dollar sign '$' indicate the start and end of a row.

For example, '^ cat' indicates that a row starting with cat needs to be matched. I hope you will understand the regular expression as follows: do not consider '^ cat' as "matching rows starting with Cat ", instead, it is understood that "C is matched as the first character of a line, followed by a A, followed by a t text.

(Note: I will enclose the regular expression ''later)

There is no difference between the above two understandings, but it is easier to understand the internal logic of Regular Expressions by character. What's special about the delimiters and dollar signs is that they match a location rather than a specific text. How does egrep explain '^ cat $', '^ $', and a single '^?

'^ Cat $' indicates that the match starts with a line (obviously, each line has a beginning), then the CAT letter, and then the end of the line.
Meaning: contains only cat rows-no additional words, blank characters ...... Only cat.
'^ $' Indicates that the match starts with the line and ends with the end of the line.
It indicates a blank line (no characters, including blank characters ).
'^' Indicates that the matching condition starts with a row.
Application significance: meaningless! Because each row has a beginning, each row can match-empty rows are no exception.

Example 1:

            var reg=/^[abc][abc][0123456789]/;            var result1=reg.exec("ca4at");            var result2=reg.exec("1catg");            document.write(result1+" "+result2);

Result:

　　

Example 2:

            var reg=/cat$/;            var result1=reg.exec("cat");            var result2=reg.exec("catg");            document.write(result1+" "+result2);

Result:

　　

Example 3:

            var reg=/^cat$/;            var result1=reg.exec("cat");            var result2=reg.exec("catcat");            document.write(result1+" "+result2);

Result:

　　

　　2.2egrep metacharacters []-

If the word to be searched is "gray" and you cannot determine whether it is "gray", you can use the regular expression struct '[]', also known as a character group. Therefore, we can write 'gr [AE] Y'. Note that only one character group can be matched successfully. The order in the character group is generally random.

If we want to match any number from 1 to 6, we can write '[123456]', we can also use the character group metacharacters '-' (hyphen, that is, '[1-6]', the same effect. '[0-9]' and '[A-Z]' are common methods to match numbers and lowercase letters. Multiple ranges are also allowed, for example, '[0-9a-za-f]'. We can also combine the character range with common text as we like: '[0-9a-z _!.?] 'Can match a number, uppercase letter, underline, exclamation point, point number, or question mark.

Note that only within the character group can be metacharacters -- otherwise, it can only match common hyphens. This may be because if a hyphen appears at the beginning of a character group, it only matches a common hyphen, rather than a range. Similarly, question marks and periods are generally treated as metacharacters, but not in character groups. For example, '[-a-Z]' is a hyphen or lowercase letter.

Example 1:

            var reg=/gr[ae]y/;            var result1=reg.exec("grygrey");            var result2=reg.exec("grygraey");            document.write(result1+" "+result2);

Result:

　　

Example 2:

            var reg=/^[abc][abc][0123456789]/;            var result1=reg.exec("ca4at");            var result2=reg.exec("1catg");            document.write(result1+" "+result2);

Result:

　　

Example 3:

            var reg=/^[a-z]/;            var result1=reg.exec("c24at");            var result2=reg.exec("1catg");            document.write(result1+" "+result2);

Result:

　　

Example 4:

            var reg=/^[0-9a-z_]$/;            var result1=reg.exec("_");            var result2=reg.exec("catcat");            document.write(result1+" "+result2);

Result:

　　

Example 5:

            var reg=/^[-!][0-9][a-z]/;            var result1=reg.exec("-7fff");            var result2=reg.exec("!75fff");            document.write(result1+" "+result2);

Result:

　　

　　2.3egrep metacharacters [^]

Replace "[]" with "[^]". This character group matches any unlisted characters, which are called excluded character groups. We noticed that the '^' here is the same as the Escape Character of the first line. The characters are indeed the same, but the meaning is completely different, the character group must be followed by the first square brackets of the character group before it represents a metacharacter.

If we use 'q [^ u] 'to match the string Iraqi "Iraq", will it succeed? Here we need to emphasize the next character group. Even the excluded character group, we need to match one character. The above match will fail because Q is not followed by any character, and '[^ u]' must match one character, whether it is a space, line break, or other word, there must be at least one.

Example 1:

            var reg1=/[^a-z]/;            var result1=reg1.exec("a24at");            var result2=reg1.exec("catg");            document.write(result1+" "+result2);

Result:

　　

Example 2:

            var reg1=/^[0-9a-z_^]$/;            var reg2=/^[^^0-9]/;            var result1=reg1.exec("^");            var result2=reg1.exec("catcat");            var result3=reg2.exec("-7fff");            var result4=reg2.exec("^75fff");            document.write(result1+" "+result2+" "+result3+" "+result4);

Result:

　　

Example 3:

            var reg=/q[^0-9]/;            var result1=reg.exec("a2qat");            var result2=reg.exec("Iraq");            document.write(result1+" "+result2);

Result:

　　

　　2.4egrep metacharacters.

Metacharacter '.' is a simple way to match character groups of any character (except for line breaks. If we need to use a placeholder "match any character" in the expression, it is very convenient to use the DoT number. For example, we need to search for 2013/01/04, 2013-01-04, or 2013.01.04. We can use '123. 01.04 or 2013 [-. /] 01 [-. /] 04 [-. /] ', but which one is better?

In '1970 [-. /] 01 [-. /] 04 [-. /] 'is not a metacharacter because it is inside the character group (remember that the definition and meaning of metacharacters are different between the character group and the character group ). The hyphens here are also not characters because they are all followed by [or [^. When the dot is a metacharacter, it can match any character, so it can also match a string such as "2013301504 [-. /] 01 [-. /] 04 [-. /] 'is more accurate, but harder to read and write. '192. 100' is easier to understand, but not detailed enough.

Example:

            var reg=/2013.01.04/;            var result1=reg.exec("reg2013-01-04");            var result2=reg.exec("2013-1-04");            document.write(result1+" "+result2);

Result:

　　

　　2.5egrep metacharacters | ()

'|' Is a very simple metacharacter, which means "or ". Let's look back at the 'gr [EA] y' example. We can also write 'Gray | gray 'or 'gr (E | A) y '. The latter uses parentheses to define the range of the Multi-choice structure (in normal cases, parentheses are also metacharacters). For the expression 'gr (E | A) y', parentheses are required, otherwise, it becomes 'gre | ay', which indicates matching GRE or ay.

'Jeffrey | Jeffery ', 'jeff (Rey | ERY)', and 'jeff (Re | ER) y' are equivalent. The example of GR [EA] y' and 'gr (E | A) y' may make people feel that the multi-choice structure is not much different from the character group, but do not confuse these two concepts. A character group can only match a single character in the target text. Each multiple-choice structure may be a complete regular expression and can match any length of text.

Example 1:

            var reg=/grey|gray/;            var result1=reg.exec("reggray");            var result2=reg.exec("greay");            document.write(result1+" "+result2);

Result:

　　

Example 2:

            var reg=/(ca)t/;            var result1=reg.exec("ttcatty");            var result2=reg.exec("ctat");            document.write(result1+" "+result2);

Result:

　　

Example 3:

            var reg=/gr(e|a)y/;            var result1=reg.exec("sagrey");            var result2=reg.exec("greay");            document.write(result1+" "+result2);

Result:

　　

　　2.6egrep metacharacters? + *

Now let's look at the matching of color and color. The difference between them is that there is more U Than the previous word. Can we use 'U U? R' to solve this problem. Metacharacters '? 'Indicates the option. Let's take a look at 'four (th )? ', Here '? 'The element is the whole bracket. The expressions in the brackets can be any complicated, but they are a whole "from the outside of the brackets.

'+' (Plus sign) and '*' (asterisks) Act similarly to question marks. Metacharacters '+' indicates that "the elements that are adjacent to each other appear once or multiple times", and '*' indicates that "the elements that are adjacent to each other appear multiple times or not ". Question marks, plus signs, and star numbers are collectively referred to as quantifiers because they limit the number of times the elements are reproduced.

Next, let's look at an HTML tag like <HR · size = 14>, which represents a horizontal line that spans the screen at a height of 14 pixels. There may be any number of spaces before the end of the angle brackets. In addition, there may also be any number of spaces on both sides of the equal sign. At last, there must be at least one space between HR and size. So we get '<HR · + size · * = · * 14 · *> '. If the number 14 of the tag we are looking for is arbitrary, you can use '<HR · + size · * = * [0-9] + · *> '.

(Note: '.' is used to indicate a space)

What if we want the size to be dispensable? Change to '<HR (· + size · * = · * [0-9] + )? · *> 'To achieve the effect. Summary: Question mark, asterisk, and plus sign:

　　

Example 1:

            var reg=/colou?r/;            var result1=reg.exec("color");            var result2=reg.exec("colouur");            document.write(result1+" "+result2);

Result:

　　

Example 2:

            var reg=/four(th)?/;            var result1=reg.exec("fourth");            var result2=reg.exec("fourt");            var result3=reg.exec("fuorth");            document.write(result1+" "+result2+""+ result3);

Result:

　　

Example 3:

            var reg=/ab+/;            var result1=reg.exec("abbbc");            var result2=reg.exec("abdb");            document.write(result1+" "+result2);

Result:

　　

Example 4:

            var reg=/(abc*)/;            var result1=reg.exec("abcccdd");            var result2=reg.exec("abdb");            var result3=reg.exec("acc")            document.write(result1+" "+result2+" "+result3);

Result:

　　

Example 5:

            var reg=/(ab *f)/;            var result1=reg.exec("ab   fdd");            var result2=reg.exec("abfdb");            var result3=reg.exec("acc")            document.write(result1+" "+result2+" "+result3);

Result:

　　

　　2.7egrep metacharacters

Some versions of egrep support the use of metacharacters to customize the interval "{min, max}" for the number of retries, which is called "interval quantifiers ". Someone uses '[A-Za-Z] {}' to match the U.S. stock code (1 to 5 letters ), the interval quantifiers corresponding to the question mark are "{0, 1 }". Javascript also supports this form.

Example 1:

            var reg=/abc{3}/;            var result1=reg.exec("abcccccdd");            var result2=reg.exec("abccdb");            document.write(result1+" "+result2);

Result:

　　

Example 2:

            var reg=/abc{1,}/;            var result1=reg.exec("abcccdd");            var result2=reg.exec("abdb");            document.write(result1+" "+result2);

Result:

　　

Example 3:

            var reg=/abc{0,1}/;            var result1=reg.exec("abcccdd");            var result2=reg.exec("abdb");            var result3=reg.exec("acc")            document.write(result1+" "+result2+" "+result3);

Result:

　　

　　2.8egrep metacharacters \ 1

So far, we have seen brackets that can limit the range of multiple choices and combine several characters into a unit. Brackets also have a useful feature-memorizing text. The text written in parentheses is in \ n. Here \ n is not a line break, and N is a specific number, for example, '([A-Z]) ([0-9]) in \ 1 \ 2', \ 1 records a lowercase letter and \ 2 records a digit.

Example:

            var reg=/a(b)c\1/;            var result1=reg.exec("abcbccccdd");            var result2=reg.exec("accdb");            document.write(result1+" "+result2);

Result:

　　

　　2.9egrep metacharacters (? :)

This is a non-capturing parentheses and is only used to separate characters and form a unit. It is not used to record. Because '()' records internal matching information, it takes up part of the memory, you can use '(? :) ', For example,' AB (? : C) D (EF) g \ 1' must match the string... Abcdefgef ..., Instead... Abcdefgc ....

Example:

            var reg=/ab(?:c)d(ef)g\1/;            var result1=reg.exec("11abcdefgef22");            var result2=reg.exec("abcdefgc");            document.write(result1+" "+result2);

Result:

　　

　　2.10egrep metacharacters \

If we need to match a certain character itself as a metacharacter, we need to use '\'. For example, if we have mentioned a match between 3.03.01.04, We need to write '2017 \. 01 \. 04 '. If the backslash is not followed by metacharacters, it must be determined by the genre and version, for example, in some egrep versions, '\ <' and '\>' are used to provide the Left and Right bounders of words. In JavaScript, '\ B' is used ', both genres also support the '\ 1' format.

Example:

            var reg=/\([a-zA-Z]+\)/;            var result1=reg.exec("tt(cat)ty");            var result2=reg.exec("c(tat");            document.write(result1+" "+result2);

Result:

　　

This section describes common simple metacharacters. However, in each language, metacharacters are different. You need to pay attention to them.

In fact, these are the most basic regular expressions used. They can only be an entry point, mainly for friends who are new to regular expressions. The core of a regular expression is the Regular Expression Engine. It is used to improve the efficiency of our expressions.