Measure the test taker's knowledge about how to calculate the reading volume of a blog.

Measure the test taker's knowledge about how to calculate the reading volume of a blog.

Content: Use wget to batch download the casual pages of my own blog park to see if the reading volume has increased
Environment: kali + python
Idea: 1. Execute the python script by calling the script on the terminal, for example, python add_readcounts.py-f my_blogs
2. You are not familiar with crawlers, but cannot add each link to the code. You can use a text file to save it and obtain a file variable.
3. Run system commands in python

1 import optparse 2 3 # get the file that user input, return the open file, get the file name and open it through Terminal input, return a file variable 4 def get_file (): 5 parser = optparse. optionParser ("usage % prog" + "-f <link_file>") # Run the script in the format of 6 parser. add_option ('-F', dest = 'fname', type = 'string', help = 'specify link file') # Add the variable 7 (options, args) for running the script) = parser. parse_args () # Save the variables obtained by the terminal. I don't know how to explain them here. I 'd like to explain it 8 if options. fname = None: # The file location parameter is incorrect. The output is used. The program ends at 9 print parser. usage10 file_name = options. fname11 # file_name = 'test' 12 f = open (file_name, 'R') 13 return f14 15 # execute the shell commands to execute the command, I tested several METHODS 16 # import subprocess17 import commands, os18 def execute_shell (s): 19 # obj = subprocess. popen ('wget', shell = True, stdout = subprocess. PIPE) 20 # x = obj. stdout. read () This is the first method. It is a bit of a problem because 21 shell_command = ''is not used if you are not familiar with it ''. join (['wget', '-O', 'x', s]) # Because wget is a download page, I write every result to a file, this will not generate many webpages 22 # print (shell_command) 23 OS. system (shell_command) # Run the command 24 # print (commands. getstatusoutput (shell_command) 25 # (status, result) = commands. getstatusoutput (shell_command) # This method will be blocked, so 26 # print (result) is not used) 27 28 29 30 import re31 if _ name _ = '_ main _': 32 f = get_file () 33 run_num = 034 for s in f: 35 # s = re. sub (';', ', s) 36 # print (s) 37 execute_shell (s) 38 run_num + = 139 40 print 'The website is:', run_num

View Code

I forgot to put my text file up. Now I want to add it.

File Name: my_blogs

Content

Http://www.cnblogs.com/-nbloser/p/7854170.html
Http://www.cnblogs.com/-nbloser/p/7873562.html
Http://www.cnblogs.com/-nbloser/p/7901274.html
Http://www.cnblogs.com/-nbloser/p/7901295.html
Http://www.cnblogs.com/-nbloser/p/7979969.html

......