Python -- Regular Expression (2)
3. Use Regular Expressions
Now we have learned some simple regular expressions, But how should we use them in Python? The re module provides an interface to connect to the Regular Expression Engine, allowing you to compile the RE into objects and use them for matching.
Certificate ------------------------------------------------------------------------------------------------------------------------------------------------------
3. 1. Compile a regular expression
Regular Expressions are compiled into a pattern object. This object has many methods for various operations, such as searching by pattern matching or replacing strings.
>>> import re>>> p = re.compile('ab*')>>> pre.compile('ab*')The re. compile () method can also accept an optional flags parameter, which is used to specify various special functions and syntax changes. We will learn it one by one later. Now let's look at a simple example:
>>> p = re.compile('ab*',re.IGNORECASE)The regular expression is passed as a string parameter to the re. compile () method. Because regular expressions are not the core part of Python and do not have special syntaxes to express them, they can only be processed as strings (many applications do not need to RE, so there is no need to include RE in the core part of Python). On the contrary, the regular expression re module is only embedded in Python as the C extension module, just like the socket or zlib module.
Using Regular Expressions As strings makes Python more concise, but it also has a disadvantage. Let's talk about it later.
Certificate ------------------------------------------------------------------------------------------------------------------------------------------------------
3. 2. troublesome backslash
As described above, regular expressions use backslash \ To make some characters have special meanings (for example, \ s) or remove the special meanings of special characters (for example, \ * indicates an asterisk without special meanings). This will conflict with characters that implement the same functions in Python strings.
If we want to write a RE to match a string "\ section" in the LaTeX file, first write the string to be matched in the program code. Next, you need to add a backslash before the backslash and other metacharacters to remove their special meanings. Therefore, the result "\ section" is obtained, and this string will be passed to re. compile () function. However, you need to know that the backslash In the Python string also has a special significance, so you need to add a backslash before the two backslash again to get the string "\\\\ section ".
| Match string |
Matching steps |
| \ Section |
String to be matched |
| \ Section |
'\' Is used in the regular expression to indicate '\' |
| "\\\\ Section" |
In the Python string, '\' is also used to represent '\' |
In short, to match a backslash character '\', you need to write four backslash '\' as a regular expression string, because the regular expression must be a double slash \\, each backslash must also be represented by the slash '\' in the Python string. This causes us to repeat the backslash many times and make the final regular expression string hard to understand.
The solution is to use the original string in Python. The so-called original string, that is, adding the letter r at the top of the string, so that the backslash in the string will be removed from the special syntax, as a normal character. For example, the string r "\ n" is a string containing '\' and 'n', while the string "\ n" is a string with only one line break. Regular Expressions are usually represented by the original strings in Python.
| Regular Expression string |
Original string |
| "AB *" |
R "AB *" |
| "\\\\ Section" |
R "\ section" |
| "\ W + \ s + \ 1" |
R "\ w + \ s + \ 1" |
Certificate ------------------------------------------------------------------------------------------------------------------------------------------------------
3. perform matching
After compiling a regular expression, you will get a schema object. What will you do with it? The Mode object contains many methods and attributes. Here we will only introduce the most commonly used methods. You can view the complete list by viewing the documents of the re module.
| Method/attribute |
Function |
| Match () |
Determine whether a regular expression is fromStartMatch a string |
| Search () |
Scan a string to find the first position matched by the regular expression. |
| Findall () |
Scan a string, find all the substrings that match the regular expression, and return them as a list |
| Finditer () |
Scan a string, find all the substrings that match the regular expression, and return them as iterators. |
If no matching string is found, the match () and search () Methods return None. If the match succeeds, a matching object will be returned, including the matching information: the start position and the matched substring.
You can use the re module in interactive mode to learn the content. If you can use tkinter, you can take a look at the tool/demo/redemo. py program, which is a sample program released with Python. It allows you to input Regular Expressions and strings, and output whether the two match. When you test a complex regular expression, redemo. py is very useful. Phil Schwartz's Kodos is also a useful interactive tool for developing and testing regular expressions.
We use the standard Python interpreter to explain these examples. First, open the Python interpreter, import the re module, and compile a RE:
>>> import re>>> p = re.compile('[a-z]+')>>> pre.compile('[a-z]+')Now you can use the regular expression [a-z] + to match various strings. However, an empty string cannot be matched because the plus sign + indicates repeated more than once. In this case, the match () method returns None. In addition, this result is not output in the interpreter, but you can call the print () method explicitly to output this result.
>>> p.match('')>>> print(p.match(''))NoneNext, let's try a string that can be matched, such as the string "tempo ". In this case, the match () method returns a match object. To use this object later, you should save the result in a variable.
>>> m = p.match('tempo')>>> m<_sre.SRE_Match object; span=(0, 5), match='tempo'>
Now you can use the matching object to query the matching string information. The matching object instance also has some methods and attributes. The most important ones are listed here:
| Method/attribute |
Function |
| Group () |
Returns the matched string. |
| Start () |
Returns the starting position of string matching. |
| End () |
Returns the end position of string matching. |
| Span () |
Returns a ancestor indicating the matched position (START, end) |
Try the following examples to quickly understand these methods:
>>> m.group()'tempo'>>> m.start(),m.end()(0, 5)>>> m.span()(0, 5)
The group () method returns a substring determined by the position of RE. start () and RE. end. The span () method returns the start position and end position of the sub-string with a ancestor. However, the match () method is used to determine whether a regular expression matches a string from the start point. Therefore, the start () method always returns 0. However, the search () method is different. It scans the entire string and the starting position of the matched substring is not necessarily 0.
>>> print(p.match(':::message'))None>>> m = p.search(':::message')>>> print(m)<_sre.SRE_Match object; span=(3, 10), match='message'>>>> m.group()'message'>>> m.span()(3, 10)In actual programs, the most common method is to store the matching object in a variable and check whether it is None. As shown below:
p = re.compile(…)m = p.match(‘string goes here’)if m:print(‘Match found: ’ , m.group())else: print(‘No match’)
There are two methods to return all matched substrings. The findall () method returns a list Of all matched strings:
>>> p = re.compile('\d+')>>> p.findall('12 drummers drumming, 11 pipers piping, 10 lords a-leaping')['12', '11', '10']The findall () method needs to create the entire list before it returns the result. However, the finditer () method returns the matching object as an iterator (Note: The iterator method saves more memory and improves efficiency ).
>>> iterator = p.finditer('12 drummers drumming, 11 pipers piping, 10 lords a-leaping')>>> iterator
>>> for match in iterator:print(match.span())(0, 2)(22, 24)(40, 42)
Certificate ------------------------------------------------------------------------------------------------------------------------------------------------------
3. 4. Module-level functions
You do not need to create a matching object to call it. The re module also provides global functions such as match (), search (), findadd (), and sub. The first parameter of these functions is a regular expression string. Other parameters use the same parameter as the method of the pattern object with the same name, and the return value is the same. Both return None or match object.
>>> print(re.match(r'From\s+','Fromage amk'))None>>> print(re.match(r'From\S+','Fromage amk'))<_sre.SRE_Match object; span=(0, 7), match='Fromage'>>>> print(re.match(r'From\S+','Fromage amk').group())Fromage>>> re.match(r'From\s+','From amk Thu May 14 19:12:10 1998')<_sre.SRE_Match object; span=(0, 5), match='From '>
In fact, these functions simply create a schema object for you and can call its related functions. In addition, it stores compiled mode objects in the cache, So if you use the same regular expression later, you don't need to re-create the mode, and you can implement quick calls.
So you should use these module-level functions? Or should we compile the method to get the mode object and then call it? If you want to use a regular expression in a loop, compiling it in advance can save function calls. However, because of the internal buffer mechanism, the efficiency of the two is not bad.
Certificate ------------------------------------------------------------------------------------------------------------------------------------------------------
3. 5. Compilation flag
The compile flag allows you to change the working method of the regular expression in some ways. There are two available names for the compilation mark in the re module: Full name and simple writing. For example, the abbreviation of IGNORECASE is letter I, you will know that the Perl language is abbreviated as this, such as re. VERBOSE and short write are re. X ). Multiple compilation flags can be logically connected. For example, re. I | re. M sets the I and M flags.
The following table lists some available compilation flags:
| Compilation flag |
Description |
| ASCII, |
Make the escape symbols such as \ w, \ B, \ s and \ d match only ASCII characters |
| DOTALL, S |
So that the dot. can match any symbols, including line breaks |
| IGNORECASE, I |
Case Insensitive |
| LOCALE, L |
Supports current language (region) settings |
| MULTILINE, M |
Multi-row matching will affect ^ and $ |
| VERBOSE, X (for 'extended ') |
Enable detailed Regular Expressions |
I
IGNORECASE
Matching is case-insensitive, so that character classes and text strings are case-insensitive when matching characters. For example, [A-Z] also matches lowercase letters, and Spam matches Spam, spam, and spAM. If you do not set the LOCALE flag, you will not consider the language (region) setting method.
L
LOCALE
So that \ w, \ W, \ B, and \ B depend on the current language environment, rather than the Unicode database.
Region settings are a function of the C language library, mainly to consider the differences in languages when writing programs. For example, if you are processing a French text, you want to write \ w + to match the word, but \ w only matches the word that appears in the character class [a-zA-Z, it won't match 'é' or '? ', If your system is set to a French language environment, the C language function will think that 'A' is also a letter. When the LOCALE flag is set during regular expression compilation, \ w can recognize French, but its speed is relatively slow.
M
MULTILINE
(^ And $ are described later)
Normally, ^ matches only the beginning of the string, while $ matches only the end of the string. When this flag is set, the metacharacter ^ matches the beginning of each row of the string, similarly, the metacharacters $ match the end of each row.
S
DOTALL
Make the dot '.' match all characters, including line breaks. If this flag is not set, the dot '.' matches all characters except the line break.
A
ASCII
Make \ w, \ w, \ B, \ B, and \ S values match ASCII characters rather than Unicode characters. This flag only makes sense for the Unicode mode and ignores the byte mode.
X
VERBOSE
This flag makes it more flexible to organize regular expressions, so that the written regular expressions are more readable. If this flag is set, spaces in the regular expression will be ignored. Of course, spaces in character classes are not included here, or spaces escaped by backslash, this allows you to organize regular expressions more clearly. In addition, this flag allows you to use comments in regular expressions. The # and subsequent characters are ignored by regular expressions, unless the pound sign # Is In The character class or is escaped by a backslash.
Here is an example of using re. VERBOSE:
>>> Charref = re. compile (r''' & [#] # Start digit reference (0 [0-7] + # octal format | [0-9] + # decimal format | x [0-9a-fA-F] + # hexadecimal format ); # end with a semicolon ''', re. VERBOSE)
If you do not need to set VERBOSE, this regular expression will be in the following format:
>>> charref = re.compile('&[#](0[0-7]+' '|[0-9]+' '|x[0-9a-fA-F]+);')In the above example, we use the Python function to automatically concatenate strings, so that the regular expression is divided into several smaller parts, but it still does not use re. VERBOSE version RE is easy to understand.