Regex Regular Expression
- 引入 re 模組
- 規則定義 patternName = r"abc..."
1. 概念 - Regex(RE)是一種小型的 高度專業化的語言
- 它內嵌在python中, 通過re模組實現
2. 作用 處理字串.
- 匹配
- 替換
- 分隔
3. 字元匹配 - 一般字元
- 元字元
. ^ $ * + ? {} [] \ | ()
# 匹配一般字元
>>> import re
>>> pattern = r"ab"
>>> re.findall(pattern, "123abc")
['ab']
4. 元字元 4.0 .
- 任一字元
4.1 []
- 在字元序列中選擇一個
- 常用來指定一個字元集: [abc], [0-9], [a-zA-Z]
- 元字元在字元集中當做一般字元處理: [abc$]
- 補集 : [^a-z]
>>> import re
# 字元集
>>> pattern = "[a-z]"
>>> re.findall(pattern, "abc")
['a', 'b', 'c']
# 補集
>>> pattern = "[^a-z]"
>>> re.findall(pattern, "abc")
[]
# 特殊字元
>>> pattern = "[a^$]"
>>> re.findall(pattern, "abc^$")
['a', '^', '$']
4.2 ^
- 匹配行首
>>> pattern = "^a"
>>> re.findall(pattern, "baaa")
[]
>>> re.findall(pattern, "abbb")
['a']
4.3 $
- 匹配行尾
>>> pattern = "a$"
>>> re.findall(pattern, "aaab")
[]
>>> re.findall(pattern, "bbba")
['a']
4.4 \ - 逸出字元
- 取消元字元的特殊含義, 將其當成一般字元處理
- 特殊含義
- \d <==> [0-9] , 匹配十進位數, decimal
- \D <==> [^0-9], 匹配非數字字元
- \s <==> [\t\n\r\f\v] , 匹配空白字元
- \S <==> [^\t\n\r\f\v]
- \w <==> [a-zA-Z0-9_], 匹配 字母 數字 底線
- \W <==> [^a-zA-Z0-9_]
4.5 重複
4.5.1 *
- 重複次數: [0, +無窮)
>>> pattern = r"ab*"
>>> re.findall(pattern, "a")
['a']
>>> re.findall(pattern, "ab")
['ab']
>>> re.findall(pattern, "abb")
['abb']
>>> re.findall(pattern, "abbbbbbbbbb")
['abbbbbbbbbb']
4.5.2 +
- 重複次數: [1, +無窮)
>>> pattern = r"ab+"
>>> re.findall(pattern, "a")
[]
>>> re.findall(pattern, "ab")
['ab']
>>> re.findall(pattern, "abbbbbb")
['abbbbbb']
4.5.3 ?
- 重複次數: [0, 1] , 即 有 或 沒有
>>> pattern = r"ab?"
>>> re.findall(pattern, "a")
['a']
>>> re.findall(pattern, "ab")
['ab']
>>> re.findall(pattern, "abbbbb")
['ab']
4.5.4 {m,n}
- {m,n} 重複次數: [m, n]
- {m} 重複次數: m
- {m,} 重複次數: [m, +無窮)
- m預設值為0
>>> pattern = r"\d{1,3}"
>>> re.findall(pattern, "1234")
['123', '4']
>>> pattern = r"\d{1,}"
>>> re.findall(pattern, "1234")
['1234']
>>> pattern = r"\d{1}"
>>> re.findall(pattern, "1234")
['1', '2', '3', '4']
5. 編譯Regex 5.1 編譯
- re模組 提供了 一個Regex引擎介面,
可以將 REstring 編譯成對象
>>> import re
>>> telPatternString = r"\d{3}"
>>> telPattern = re.compile(telPatternString)
>>> telPattern
<_sre.SRE_Pattern object at 0x01806170>
>>> telPattern.findall("1")
[]
>>> telPattern.findall("123")
['123']
>>> telPattern.findall("1234")
['123']
5.2 編譯時間 使用參數
- 忽略大小寫
>>> import re
>>> namePatternString = r"[a-z]{3}"
>>> namePattern = re.compile( namePatternString, re.IGNORECASE )
>>> namePattern.findall("abc")
['abc']
>>> namePattern.findall("abC")
['abC']
5.3 反斜線的麻煩
- 字串前加"r", 反斜線就不會被任何特殊方式處理
>>> pattern = r"\\"
>>> re.findall(pattern, "c:\dirA")
['\\']
>>> pattern = "\\"
>>> re.findall(pattern, "c:\dirA")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "D:\Python\install-2.7\lib\re.py", line 177, in findall
return _compile(pattern, flags).findall(string)
File "D:\Python\install-2.7\lib\re.py", line 242, in _compile
raise error, v # invalid expression
sre_constants.error: bogus escape (end of line)
6. Regex object 的一些方法 - match() 只匹配開頭的符合規則的字串, 失敗-返回None
- search() 匹配任意位置的符合規則的字串
- findall() 將 符合規則的字串 作為 list返回
- finditer() 將 符合規則的字串 作為 迭代器返回
- sub() 替換, 返回替換後的字串
- subn() 替換, 返回(替換後的字串, 替換的次數)
- split() 切割
6.1 match()
>>> import re
>>> pattern = r"a"
>>> re.match(pattern, "abc")
<_sre.SRE_Match object at 0x01407FA8>
>>> re.match(pattern, "ba")
>>> re.match(pattern, "bac")
>>>
6.2 search()
>>> import re
>>> pattern = r"a"
>>> re.search(pattern, "abc")
<_sre.SRE_Match object at 0x01419330>
>>> re.search(pattern, "bac")
<_sre.SRE_Match object at 0x01407FA8>
>>> re.search(pattern, "bca")
<_sre.SRE_Match object at 0x01419330>
>>>
6.3 findall()
>>> import re
>>> pattern = r"a"
>>> re.findall(pattern, "abacad")
['a', 'a', 'a']
6.4 finditer
>>> import re
>>> pattern = r"[0-9]"
>>> re.finditer(pattern, "1234")
<callable-iterator object at 0x017FE990>
>>> for x in re.finditer(pattern, "1234") :
... print x
...
<_sre.SRE_Match object at 0x01407FA8>
<_sre.SRE_Match object at 0x01419330>
<_sre.SRE_Match object at 0x01407FA8>
<_sre.SRE_Match object at 0x01419330>
>>>
6.5 sub() subn()
- subn(pattern, repl, string, count=0, flags=0)
>>> re.sub(r"a", "x", "abca")
'xbcx'
>>> re.subn(r"a", "x", "abca")
('xbcx', 2)
6.6 split()
- split(pattern, string, maxsplit=0, flags=0)
>>> re.split("[^\d]", "1999-09/19 23:34:59")
['1999', '09', '19', '23', '34', '59']
>>> re.split("[^\d ]", "1 + 2 + 3 - 4 * 5")
['1 ', ' 2 ', ' 3 ', ' 4 ', ' 5']
7. Match object 的一些函數 - group() 返回被正則匹配的字串 obj.group()
- start() 匹配字串的起始位置
- end() 匹配字串的末尾位置
- span() (起始位置, 末尾位置)
- 檢查 Match object 是否為 None, 判斷是否 匹配成功.
8. re屬性 - 編譯標識 flags
- DOTALL/S 使匹配包括換行在內的所有字元
- IGNORECASE/I 忽略大小寫
- LOCALE/L 本地化匹配
- MULTILINE/M 多行匹配, 影響 ^$
- VERBOSE/X 去除"""編寫正則時的分行符號
# re.S
>>> re.findall(r"a.b", "a\nb")
[]
>>> re.findall(r"a.b", "a\nb", re.S)
['a\nb']
# re.M
>>> s = """
... line1: a1
... line2: a2
... line3: a3
... """
>>> s
'\nline1: a1\nline2: a2\nline3: a3\n'
>>> re.findall(r"^line[0-9]", s)
[]
>>> re.findall(r"^line[0-9]", s, re.M)
['line1', 'line2', 'line3']
# re.X
>>> telPatternStr = r"""
... \d{3,4}
... -?
... \d{7}
... """
>>> telPatternStr
'\n\\d{3,4}\n-?\n\\d{7}\n'
>>> re.findall(telPatternStr, "011-1234567")
[]
>>> re.findall(telPatternStr, "011-1234567", re.X)
['011-1234567']
9. 正則 分組 - () - ( pattern1 | pattern2 ) 二選一
- 分組優先被返回
# 耙梳址
>>> s = """
... <a href="www.baidu.com">baidu</a>
... <a href="www.sina.com.cn">sina</a>
... """
>>> print s
<a href="www.baidu.com">baidu</a>
<a href="www.sina.com.cn">sina</a>
>>> re.findall( r"<a href=\".+\">.+</a>", s )
['<a href="www.baidu.com">baidu</a>', '<a href="www.sina.com.cn">sina</a>']
>>> re.findall( r"<a href=\"(.+)\">.+</a>", s )
['www.baidu.com', 'www.sina.com.cn']
>>>
10. 小爬蟲 - 下載 貼吧或QQ空間中 所有圖片
- GrapPicture.py
'''Created on 2013-10-4@author: WuQinfei'''import reimport urllib# url : web site# return : get src code from the URLdef getHtml(url) : page = urllib.urlopen(url) # connect to the url html = page.read() # read it return html # return src code# html : html src code# return : a list of jpg URLsdef getImg(html) : reg = r'src="(http://[^\s]*\.jpg)" width' imgRe = re.compile(reg) imgUrlList = re.findall(imgRe, html) return imgUrlList# url : download by this url# name : saved by this name in current dir def downByUrl(url, name) : urllib.urlretrieve(url, name)################################################if __name__ == "__main__" : html = getHtml("http://tieba.baidu.com/p/2306540022") imgUrlList = getImg(html) count = 1 stopNum = 10 for imgUrl in imgUrlList : print "download....", imgUrl pictureName = "E:\\desktop\\python\\py_src\\jpg\\%s.jpg" % count downByUrl(imgUrl, pictureName) count+=1 if count > stopNum : break; print "the number of pictures =", count-1
