One-stop explanation of JS Regular Expressions (from principle and syntax to JS regular expressions) and js Regular Expressions

One-stop explanation of JS Regular Expressions (from principle and syntax to JS regular expressions) and js Regular Expressions

Regular Expressions are like a beacon. When you are at a loss in the ocean of strings, you can always give you some ideas. Regular Expressions are like a money detector, when you don't know the authenticity of the money you submit, you can always identify it at a Glance. Regular Expressions are like a flashlight. When you need something, always help you get what you want...

-- Excerpt from Stinson's Chinese Comparison Sentence exercise Regular Expression

After reading a piece of literature excerpt, we will sort out the regular expressions in JS. The primary purpose of this article is to prevent me from forgetting some regular expressions, therefore, sort out and write down documents to enhance proficiency and serve as a reference. The secondary goal is to share with you. If you have any questions, please kindly advise and thank you for your patience.

Since this article is entitled "one-stop", we must be worthy of "Dragon", so it will include the regular expression principle, syntax overview, JS (ES5) in terms of regular expressions and ES6 regular expressions, and the idea of Regular Expressions in practice, I try to explain these things in a as simple as possible (as if I could really do it in a simple way ), if you only want to know how to apply the rule, you can see the second, third, and fifth parts to meet your needs. If you want to master the regular expressions in JS, then you may be wronged to follow my ideas!

I. Introduction to principles

When I started using regular expressions, I thought it was amazing. How does a computer match strings Based on a regular expression? It wasn't until I met a book called computing theory that saw the concepts of regular expressions, DFA, and NFA and their relationships that I realized.

But if you really want to understand the Regular Expression in principle, I'm afraid the best way is:
1. First, I will go to a book dedicated to regular expressions, which will be included in o'reilly's "animal story" series;
2. implement a regular engine by yourself.

The focus of this article is on the regular expression application in JS, so the principle is only a brief introduction (because I have never written about the regular expression engine, and I am not going into it ), as a result, curious babies like me are confused about the regular expression principle. Second, they will have some basic knowledge about the principle, which is of great benefit for understanding the syntax and writing regular expressions.

1. Regular Expression Engine
Why is the regular expression effective? Because there is an engine, this is the same as why javascript can be executed. There is a JS engine. The so-called regular expression engine can be understoodThe algorithm is used to simulate a machine based on your regular expression. This machine has many statuses. by reading the strings to be tested, it jumps between these statuses, if You finally stop at "Happy Ending", then Say I Do, otherwise Say You Are a Good Man. So that a regular expression is converted into a machine that can calculate the result in a limited number of steps, the engine is implemented.

The Regular Expression Engine can be roughly divided into two types: DFA and NFA
1. DFA (Deterministic finite automaton) Deterministic finite automaton
2. NFA (Non-deterministic finite Automation) Non-deterministic finite automaton, most of which are NFA

The "deterministic" indicates that for a certain character input, the status of this machine will jump from a to B, and the "non-deterministic" indicates, for a certain character input, this machine may have several status hops. Here, the "poor" means that the status is limited, you can determine whether a string is accepted or havip card in a limited number of steps. The "Automation" here can be understood as that once the rules of this machine are set, you can judge it by yourself.

The DFA engine does not need to trace back, so the matching efficiency is usually high, but it does not support capture groups, so it does not support reverse references and references in the form of $, it also does not support some NFA engine features such as Lookaround and non-Greedy mode.

If you want to learn more about regular expressions, DFA, and NFA, you can go to computing theory and draw an automatic machine based on a regular expression.

2. Knowledge reserves

This section is useful for understanding regular expressions, especially what is a character and a position.

2.1 string in the regular eye-n characters, n + 1 position

In the above "Laughter" string, there are a total of 8 characters, you can see this, there are 9 locations, this is what smart people can see. Why do we need to have characters and positions? Because the location can be matched.

Then let's further understand "possession character" and "Zero Width ":

• If a subregular expression matches a character rather than a position and is saved to the final result, the subexpression occupies a character, for example,/ha/(matching ha) occupies characters;
• If a sub-regular matches a position rather than a character, or the matched content is not stored in the result (in fact, it can also be regarded as a position), the sub-expression is zero-width, for example,/read (? = Ing)/(match reading, but only put read into the result. The syntax will be detailed below. Here we only use this example), where (? = Ing) is zero-width, which represents a position in essence.

The possession characters are mutually exclusive, and the zero width is non-mutex. That is, a character can only be matched by one subexpression at a time, while a position can be matched by multiple zero-width subexpressions at the same time. For example, if/aa/cannot match a, a in this string can only be matched by the first a character of the regular expression, but not by the second a character at the same time ); however, the position can be matched multiple times. For example,/\ B \ ba/can match a. Although the regular expression contains two \ B metacharacters indicating the start position of a word, the two \ B can match the position 0 at the same time (in this example.

Note: The character and position are oriented to strings, while the possession character and zero width are oriented to regular expressions.

2.2 control and transmission
These two words may be used when you search for blog posts or materials. Here is an explanation:

ControlWhich Regular Expression (which may be a string of common characters, metacharacters, or metacharacters) matches a string.

TransmissionIt refers to a mechanism of the Regular Expression Engine. The drive device will locate the regular expression from where it starts matching.

When a regular expression starts to match, it usually gets control from a subexpression, starting from a position in the string to try matching, and starting from a subexpression to try matching, it starts from the end position of a successful match.

Read (? = Ing) ing \ sbook matches reading book. We regard this regular expression as five subexpressions: read ,(? = Ing), ing, \ s, and book. Of course, you can also read it as a subexpression with four separate characters, just for convenience. Read matches from position 0 to position 4, followed (? = Ing) continues from position 4 and finds that position 4 is followed by ing, so the assertion matches successfully, that is, the entire (? = Ing) is to match the position 4 (here we can better understand what is Zero Width), and then the following ing matches the position 7 from position 4, then \ s matches from position 7 to position 8, the final book matches from position 8 to position 12, and the whole match is completed.

3. Matching Tour (skip)
After talking about this, we regard ourselves as a regular expression engine, step by step taking the smallest unit-"character" and "location"-to see the regular expression matching process. Let's take a few examples.

3.1 basic match
Regular Expression: easy
Source string: So easy
Matching process: first, the control is obtained by the regular expression character e, and the matching starts from the string position 0. If the matching fails, the Regular Expression Engine will drive forward, starting from position 1 and encountering the character string 'O', the match fails, and the drive continues. The space at the end naturally fails, so the match starts from position 3, the string 'E' is successfully matched, and the control is handed over to the regular expression subexpression (Here it is also a character) a, trying to match from the last successful match ending position 4, the string 'A' is matched successfully, followed by 'y', and the matching is complete. The matching result is easy.

3.2 zero-width matching

Regular: ^ (? = [Aeiou]) [a-z] + $ source string: apple

First, this regular expression matches such a complete string from start to end, which is composed of only lowercase letters, it must start with any of the five letters a, e, I, o, and u.

Matching process: first, the regular ^ (indicating the start position of the string) gets control, starts from the position 0, matches successfully, and the control is handed over (? = [Aeiou]). This subexpression requires that the right side of the position must be one of the lower-case vowels. The zero-width subexpressions are not mutually exclusive, so the matching starts from the position 0, the right side is the string 'A', so the matching is successful, so (? = [Aeiou]) match the position 0 here. The control is handed over to [a-z] +. The match starts from the position 0. Each character in the string 'apple' is matched successfully, match to the end of the string, control returns the regular $, and attempts to match the end position of the string. Successful. At this point, the entire match is complete.

3.3 greedy match and non-Greedy match

Regular 1: {. *} regular 2 :{.*?} Source string: {233}

Here there are two regular expressions, which are added after the qualifier (what is the delimiter In the syntax? The symbol indicates that the priority quantifiers are ignored, which is non-Greedy match. I can peel this chestnut a little faster.

First, the {match at the beginning. Both regular expressions are the same.

Regular 1. * It is greedy match, so it always matches the remaining string '000000' and matches to the end position of the string. It only records an alternative state for each match. In order to trace back later, there are two paths for each match, and the matching path is selected. But remember that there are still some unmatched paths. If there is a dead end, you can return them, at this time, the control is returned to the regular expression} to match the end position of the string. If the result fails, the backtracing is performed, which means the previous one. * If you eat too much food, you can spit it out, so the control is returned. *, spit out a} (in fact, it uses the alternative status of the previous record and does not need to be tried. * match '}'), and the control is forwarded to the regular expression}. This match is successful.

Regular 2 .*? It is a non-Greedy match and matches as little as possible. Therefore, when matching every character of '000000', it tries not to match, but once the control is returned to the final}, a problem is found, quickly backtrack and match, so every character is like this, and the match is successful.

In the cloud? That's right! You can go to the recommended blog below to see it:

To learn more about the principles of greedy and non-Greedy matching and obtain more regular expressions, we recommend that you go to a CSDN blog without any trace-blog channel-CSDN. NET, detailed and thorough explanation

Ii. Syntax Overview

I believe that many people have read the 30-minute tutorials written by deerchao. I also started from that article, and deerchao started from the perspective of syntax logic. NET regular expressions are used to describe the regular syntax, And I want to organize it again, so that the syntax can be reorganized from the perspective of the application and by using JS as the host language, this allows us to translate the language description into a regular expression.

The following figure (which may need to be enlarged) sorts out common regular syntax, in addition, the unsupported syntax features of JS are marked in red (the text will not describe these unsupported features). The detailed description of the syntax section will also be based on the following figure, sort the order from top to bottom and from left to right.

1. Use some common characters-simple metacharacters

Why do we need to add two simple words here, because in regular expressions, such as \ d and \ w are called metacharacters, and {n, m },(?! Exp) This is also called metacharacters, so metacharacters are specific identifiers in regular expressions. This section describes some simple metacharacters.

• . Match any character except the line break, that is, [^ \ n]. To include any character, you can use (. | \ n)
• \ W matches any letter, number, or underline, which is equivalent to [a-zA-Z0-9 _]. In deerchao, it is also pointed out that it can match Chinese characters, but \ w cannot match Chinese Characters in JS.
• \ S matches any blank space, including page feed \ f, line feed \ n, carriage return \ r, horizontal tab \ t, vertical tab \ v
• \ D matching number
• \ Un matches n. Here, n is a Unicode character with four hexadecimal numbers. For example, \ u597d represents the Chinese character "good ", how do I express characters that exceed the \ uffff number? The u modifier of ES6 will help you.

2. to indicate the number of occurrences (repeated) -- qualifier

• A * indicates the number of consecutive occurrences of character a> = 0
• A + indicates the number of consecutive occurrences of character a> = 1
• A? It indicates that character a appears 0 or 1 times.
• A {5} indicates that character a appears five times in a row
• A {5,} indicates the number of consecutive occurrences of character a> = 5
• A {5, 10} indicates that character a appears for 5 to 10 consecutive times, including 5 and 10

3. Match location-location operator and zero-width assertion

The expressions matching a certain position are all zero-width, which mainly consists of two parts: one is the positioning operator, matching a specific position, and the other is the assertion of zero-width, match a location that meets certain requirements.

The following operators are commonly used:

• \ B matches the boundary position of a word. The exact description is that it matches a position. This position is not entirely a character that can be described by \ w, therefore, image \ u597d \ babc can match "Good abc.
• ^ Match the starting position of the string, that is, the position 0. If the Multiline attribute of the RegExp object is set, ^ matches the position after '\ n' or' \ R '.
• $ Matches the end position of the string. If the Multiline attribute of the RegExp object is set, $ matches the position before '\ n' or' \ R '.

Assertion with Zero Width (supported by JS) has the following two:

• (? = Exp) matches a position. The expression exp can be matched on the right of this position. Note that this expression only matches one position, but it only requires the right of this position, the right side of the object will not be put into the result, for example, read (? = Ing) to match "reading", the result is "read", and "ing" won't put the result
• (?! Exp) matches a position. The right side of this position cannot match the expression exp.

4. To express the meaning of "or"-character clusters and differences

We often express the meaning of "or". For example, either of these characters can be used, or five digits or five letters can be used.

Character clusters can be used to express the "or" Semantics at the character level, indicating either of the characters in square brackets:

• [Abc] indicates any of the three characters, a, B, and c. If the letters or numbers are consecutive, they can be connected, [B-f] indicates any one of the many characters from B to f.
• [(AB) (cd)] is not used to match the string "AB" or "cd", but to match a, B, c, d ,(,) one of the six characters, that is, the requirement for "matching string AB or cd", cannot be used. So we need to write AB | cd. But here we need to match the parentheses themselves, and the principle is to escape the characters by backslash. However, in square brackets, parentheses are treated as common characters. Even so, we recommend that you explicitly convert them into meanings.

Differences are used to express the "or" Semantics at the expression level, indicating matching | either of the left and right expressions can be:

• AB | cd matches the string "AB" or "cd"
• It will be short-circuited. Think back to the logic or short-circuited circuit in the programming language, so (AB | abc) is used to match the string "abc" and the result will be "AB ", because the content on the left of the vertical line is satisfied, the matching result on the left is used to represent the result of the entire regular expression.

5. I want to express the meaning of "Non ".

Sometimes we want to express the need for "except for some characters", so we need to use the negative sense.

• \ W, \ D, \ S, and \ B use uppercase letters to indicate that they are irrelevant to the matching content of lowercase letters, these match "characters except letters, numbers, and underscores", "non-digit characters", "non-blank characters", and "non-word boundary positions" in sequence"
• [^ Aeiou] indicates any character except a, e, I, o, and u. It is excluded from square brackets and appears at the beginning, if ^ does not appear in square brackets, it only represents the ^ character itself.

6. Overall View and capture-group and back-Reference

In fact, you have seen parentheses in some of the above. Yes, parentheses are used for grouping, and the parentheses are a group.

Most of the above is for character level. For example, if you want to repeat the character string "ABC" for five times, you can use A {5}. What if you want to repeat the character string "ABC" for five times? Brackets are used in this case.

The first role of parentheses is to treat the enclosed group as a whole, so you can add a qualifier after a group like a repeating character, for example (ABC) {5 }.

The content matched by the Group is the content captured by this group. From left to right, it is marked with left parentheses. Each group will automatically have a number starting from 1, the group numbered 0 corresponds to the entire regular expression. JS does not support the display name of the capture group.

The second role of parentheses, the content captured by the Group, can be later referenced in the form of \ group number. For example, (AB | cd) 123 \ 1 can match "ab123ab" or "cd123cd", but it cannot match "ab123cd" or "cd123ab". Here there is a pair of parentheses, which are the first pair of parentheses, therefore, the number is capture group 1, and then the capture content of capture group 1 is referenced by \ 1 in the regular expression. This is called backward reference.

The third role of parentheses is to change the priority. For example, abc | de and (abc | d) e are not expressed at all.

7. Escape

Escape is recommended for any character that has a function in the regular expression, even if it is not escaped in some cases, such as parentheses and ^ symbols in.

8. Priority Issues

The priority from high to low is:

• Escape \
• Parentheses (parentheses and square brackets )(),(? :),(? =), []
• Characters and locations
• Vertical bars |

9. Greedy and non-greedy

In the qualifier, except for {n}, the exact number of times is repeated, and the rest are a range with a lower limit.

In the default mode (Greedy), as many Matching content as possible. For example, if you use AB * to match the string "abbb", the result is "abbb ".

But by adding a question mark after the qualifier? Non-Greedy match can be performed to match as little as possible. Use AB *? Match "abbb" and the result is "".

A qualifier without a question mark is also called a match preference. a qualifier with a question mark is also called a ignore match preference.

10. modifier (matching options)

In fact, there are many options for regular expression matching. different host language environments may have different options. Here we will describe the JS modifier:

• G modifier: indicates global match. The pattern will be applied to all strings instead of being stopped when the first match is found.
• I modifier: Not case sensitive
• The m modifier indicates the multiline mode, which changes the behavior of ^ and $.

3. Regular Expressions in JS (ES5)

The regular expressions in JS are represented by the reference type RegExp. The following describes how to create the RegExp type, the two main methods, and the constructor attributes, and then the pattern matching in the String type, in the end, we will briefly list some limitations of the Regular Expression in JS.

1. Create a regular expression

One is to create in a literal way, and the other is to create using constructor. We always recommend that you use the former.

// Create a regular expression var exp =/pattern/flags; // For example, var pattern =/\ B [aeiou] [a-z] + \ B/gi; // equivalent the following constructor creates var pattern = new RegExp ("\ B [aeiou] [a-z] + \ B", "gi ");

Among them, pattern can be any regular expression, and the flags part is the modifier. As described above, there are three (ES5): g, I, and m ).

Now let's talk about why not use constructors, because using constructors to create regular expressions may lead to double escape of some characters. In the above example, in the constructor, the first parameter must be a string (ES6 can be a literal), so the character \ will be converted to \, so the literal \ B will be converted to \ B in the string, in this way, it is easy to make mistakes, and there are many anti-slashes.

2. Match the extracted method on RegExp -- exec ()

Var matches=pattern.exe c (str); accepts a parameter: Source string returned: Result array, and returns null if no match exists.

The result array contains two additional attributes. index indicates the position of the matching item in the string, input indicates the source string, and matches [0] indicates the string that matches the matching of the entire regular expression in the result array, matches [n] indicates the string that the nth capture group matches in the pattern.

Note that, first, exec () always returns only one matching item (that is, matching the entire regular expression), and second, if the g modifier is set, exec () is called each time () searches for new matching items in the string without setting the g modifier. If you call exec () for a string, only the first matching item is returned. Therefore, if you want to match all the places in a string that need to be matched, you can set the g modifier and call the exec method continuously through the loop.

// Match the word var str = "Reading and Writing" at the end of all ing; var pattern =/\ B ([a-zA-Z] +) ing \ B/g; var matches; while(matches=pattern.exe c (str) {console. log (matches. index + ''+ matches [0] +'' + matches [1]);} // output twice in a loop // 0 Reading Read // 12 Writing Writ

3. The RegExp method used to test whether the matching is successful or not -- test ()

var result=pattern.test(str);

Accept a parameter: The Source string
Return: Find the matching item, return true, return false if not found

4. RegExp constructor attributes

The RegExp constructor contains some attributes that apply to all regular expressions in the scope and change based on the last regular expression operation executed.

• RegExp. input or RegExp ["$ _"]: string to be matched last time
• RegExp. lastMatch or RegExp ["$ &"]: last match
• RegExp. lastParen or RegExp ["$ +"]: The last matched capture group.
• RegExp. leftContext or RegExp ["$ '"]: text before lastMatch in the input string
• RegExp. rightContext or RegExp ["$ '"]: lastMatch text in the input string
• RegExp ["$ n"]: indicates the content of the nth capture group. n is 1-9.

5. String-type pattern matching
The exec and test mentioned above are both methods on the RegExp instance. The caller is a regular expression, and the string-based call mode matching is also the most common.

5.1 match
Calling the match method on a string is essentially the same as calling exec on a regular expression, but the result array returned by the match method does not have the input and index attributes.

Var str = "Reading and Writing"; var pattern =/\ B ([a-zA-Z] +) ing \ B/g; // call matchvar matches = str on String. match (pattern); // equivalent to calling execvar matches=pattern.exe c (str) on RegExp );

5.2 return Index search Method
The accepted parameters are the same as the match method. They are either a regular expression or a RegExp object.

// The following two console outputs are the same, both of which are 5var str = "I am reading. "; var pattern =/\ B ([a-zA-Z] +) ing \ B/g; var matches=pattern.exe c (str); console. log (matches. index); var pos = str. search (pattern); console. log (pos );

5.3 find and replace the replace Method

Var result = str. replace (RegExp or String, String or Function); first parameter (Search): RegExp object or a String (this String is considered as an ordinary String) the second parameter (replace content): a string or a function returns: The replaced result string without changing the original string.

The first parameter is a string.

Only Replace the first substring

The first parameter is regular

If the g modifier is specified, all places matching the regular expression will be replaced. Otherwise, only the first place will be replaced.

The second parameter is a string.

You can use some special character sequences to insert values operated by regular expressions into them, which is very common.

• $ N: match the content of the nth capture group. n is 0-9.
• $ Nn: match the content of the nn capture group. The nn value ranges from 01 to 99.
• $ ': String matching the substring
• $ ': Match the string before the substring
• $ &: Match the entire pattern string
• $: Indicates the symbol itself.

The second parameter is a function.

• If there is only one matching item, three parameters will be passed to this function: The pattern matching item, the position of the matching item in the string, the original string
• When there are multiple capturing groups, the transmitted parameters are pattern matching items, the first capturing group, the second, and the third... the last two parameters are the match of the pattern in the string position, the original string

This function returns a string indicating the match to be replaced.

5.4 split string

Splits a string into multiple sub-strings Based on the specified delimiter and puts the results into an array. The first parameter accepted can be a RegExp object or a string (not converted to a regular expression ), the second parameter specifies the array size to ensure that the array size does not exceed the specified size.

6. Limitations of Regular Expression in JS (ES5)

The following regular expression features are not supported in JS (ES5) (as shown in the figure at a Glance ):

Match the \ A and \ Z anchor at the beginning and end of the string to Backward Search (so the assertion after Zero Width is not supported) and combine with the Unicode support of the intersection class atomic group (after \ uFFFF) annotation for matching single row and non-interval mode conditions of the named capture group

Iv. Major Enhancement of ES6 Regular Expressions
ES6 has made some enhancements to the regular expression. Here, we will simply list the following three main points. For details, refer to ES6.

1. the constructor can pass the regular expression literal.
In ES5, constructor cannot accept literal regular expressions, so there are double escaping, but ES6 supports. Even so, we recommend that you use literal expressions for simple and efficient creation.

2. u Modifier
After the u modifier is added, Unicode characters greater than \ uFFFF are correctly processed, meaning that Unicode characters of 4 bytes can also be supported.

// \ UD83D \ uDC2A is a 4-byte UTF-16 encoded, representing a character/^ \ uD83D/u. test ('\ uD83D \ uDC2A') // false. If u is added, it can be processed correctly /. test ('\ uD83D \ uDC2A') // true, without u, treated as two unicode characters

When the u modifier is added, some regular operations will be changed:

• . Originally, only characters not greater than \ uFFFF can be matched. The u modifier can be used to match any Unicode character.
• The new Unicode Character notation \ u {code point} must be valid after the u modifier is added
• After u modifier is used, all quantifiers will correctly identify Unicode characters with a point greater than 0xFFFF
• Make some antsense metacharacters take effect for characters greater than \ uFFFF

3. y Modifier

The modifier y is similar to the modifier g. It is also a global match, starting from position 0. The last match starts from the next position where the last match was successful.

The difference is that the g modifier can be matched as long as there is a match in the remaining position, and the y modifier ensures that the match must start from the first position.

Therefore, matching "ba" with/a/y will fail. As required by the y modifier, matching starts at the first position (where the position is 0) in the remaining position.

For more information about ES6 regular expressions, see this article.

5. Application of Regular Expressions

Apply regular expressions. Generally, You need to first think about regular expressions (nonsense). As long as you see the requirements related to "finding" and the source can be stringized, you can think of using regular expressions.

Generally, there are two types of applications of Regular Expressions: one is verification, and the other is search, extraction, and replacement. Verification, the most common such as form verification; search, with some set commands plus keywords to search; extraction, extracted from a paragraph of text what, or what to extract from a JSON object (because the JSON object can be stringized); replace, which is used in the template engine.

1. verification issues

Verification problems are the most common problems we encounter. In this case, we don't know what the source string looks like. The ghost knows what evil things the cute users will do, the recommended method is as follows:

1. First, describe your strings in the vernacular. After the descriptions are completed, you can use your own examples to find out what strange things users may enter, there are accepted and unacceptable (this is what you know). In this process, you may modify the previous description;
2. Split your description into regular expressions;
3. Test whether your regular expression is consistent with your expectation in the previous example. We recommend that you use the online JS regular expression test instead of writing it over and over again.

2. Search, Extraction, replacement issues

Generally, we know the format or general content of the source text. Therefore, when solving such problems, we usually have some source data for testing, we need to extract or replace from the source data.

1. Find the part of the source data you need;
2. Observe the features of these parts, the features of these parts and the features surrounding these parts. For example, the first symbol must be a comma, And the last symbol must be a colon, in short, it is to find the rule;
3. Evaluate the knowledge of the features you are looking for. First, do you know exactly what you want from the landmarks? There will be no less or more. Then, do you think about the future source data, will these features disappear in the future;
4. Organize your description of the part you are looking for and clearly describe the features you have examined;
5. Translated into a regular expression;
6. Test.

Finally, I have read and written more than 10 thousand words about the JS regular expression. After writing, I found myself skilled in the Regular Expression and made another step. Therefore, we recommend that you sort out the regular expression frequently, which is very useful and happy to share with you, thanks to everyone who can read this article.

I did not carefully review the review. If you have any problems or find any omissions, I hope you can leave a message. I will modify it in time.