Python set checkpoint simple implementation code _python

The

says checkpoints are actually records of past history, it can be thought of as log. But here's a simplification. For example, I now have a piece of text. There is a heap of links in the text. My task now is to download the contents of those addresses. In addition to the problems of the network or the website, Each download may not be very successful. There may be a broken chain or socket abnormal error. But no matter what kind of mistake, I hope my program can keep running. Or you can stop and continue running from the link for download. Instead of running from where it started. The question is very simple. Because these links are context-independent (context-sensitive situations need to be analyzed separately). So I just need to keep track of the last one before the program stops, and hopefully I can continue the work ahead. This implementation is used to record the original link, you can also use the counter method to record. The code is as follows:

# This exception is the class Checkpointmisscontenterror that does not appear in the contents of the original text  
: 
Pass # Moves the file reading pointer fd to the content corresponding to the checkpoint  
# Check The rule for point is to read a file on one or more lines, and then send this line or lines into the  
# Check file check_point. Run the program again at a later time to continue running from that checkpoint.  
def gocheckpoint (fd,check_point): 
  if not Os.path.isfile (check_point): 
    f_check = open (Check_point, ' W ') 
    f_check.close () 
  f_check = open (Check_point, ' r ') 
  lines = F_check.readlines () 
  If Len (lines) > 0: 
    check_content = lines[-1] #找到检查点最后一行  
    check_content = Check_content.strip ('/n/r ') 
    # go to check point< C16/>while True: 
      content = Fd.readline () 
      if content = = ': # EOF  
        raise Checkpointmisscontenterror 
      If Content.strip ('/n/r ') = = Check_content: Break 
         
         
  

With the above paragraph is not enough, you need the following code to add:

# Pseudo-code def Download (downloadlist,sleep_time): If Os.path.isfile (downloadlist): F = Open (downloadlist) # Check_point file name, where Check_point = File[0:file.rfind ('. ') for automatic generation of checkpoint files. + ' _check.txt ' Util.gocheckpoint (f,check_point) #这就是上面代码中的GoCheckPoint函数 F_check = open (Check_point, ' a ') # to write in append mode 
        into Try:while true:content = f.readline () if content = = ': # EOF break Content = Content.strip ('/n/r ') if Content!= ': # has download URL time.sleep (s Leep_time) Downloadoper (Path,url) #这里是伪代码. You can think of the urllib.request.retrieve () function or the # Urllib.request.urlopen () as the response to the action and then write the content to the checkpoint file F_check.write (co 
      ntent+ '/n ') F_check.flush () # must be, otherwise it will be cached, will not be written to the hard drive except: # Jumping an exception is not afraid, then press F5 again to execute raise Exception () Return Util.failure # This is the constant I set, and we think it's 0 or 1. Finally:f.close () F_check.close () # to close file pri NT (' DownloaDing are done ...................  

The

is then written to the checkpoint file after the operation is done. After the program hangs up, as long as the checkpoint file is still in, you can continue the previous work. But the checkpoint here is a bit too simple to check the transactions in the database.