How to periodically capture webpage content using Python

This article mainly introduces Python's method of periodically capturing webpage content, involving Python time functions and related operation skills of Regular Expression matching, which has some reference value, for more information about how to periodically capture webpage content using Python, see the following example. We will share this with you for your reference. The details are as follows:

1. the sched module can periodically execute specified functions.

2. Capture the specified webpage in the periodical execution of the specified function and parse the desired webpage content. The Code contains the number of online users of the six-dimensional forum.

Code for counting online forum users:

# Coding = utf-8import time, sched, OS, urllib2, re, string # initialize scheduler class of the sched Module # The first parameter is a function that can return a timestamp, the second parameter can be blocked before the scheduled time reaches. S = sched. scheduler (time. time, time. sleep) # function def event_func (): req = urllib2.Request ('HTTP: // bt.neu6.edu.cn/') response = urllib2.urlopen (req) rawdata = response. read () response. close () usernump = re. compile (R' total.*?Person online ') usernummatch = usernump. findall (rawdata) if usernummatch: currentnum = usernummatch [0] currentnum = currentnum [string. index (currentnum, '>') + 1: string. rindex (currentnum, '<')] print "Current Time:", time. strftime ('% Y, % m, % d, % H, % m', time. localtime (time. time (), 'user num: ', currentnum # Save the result for the Chart Tool amcharts to use result = open ('liuvusernum', 'A') result. write ('{year: new Date (' + time. strftime ('% Y, % m, % d, % H, % m', time. localtime (time. time () + '), value:' + currentnum + '}, \ n') result. close () # enter four parameters: interval event, priority (used for simultaneous execution of two events that arrive at the same time), and function triggered by the call, parameters (Note: tuple must be used for example, if there is only one parameter (xx,) def perform (inc): s. enter (inc, 0, perform, (inc,) event_func () def mymain (inc = 900): s. enter (0, 0, perform, (inc,) s. run () if _ name _ = "_ main _": mymain ()

I hope this article will help you with Python programming.