Write and test WordCount-Python implementation, wordcountpython

Source: Internet
Author: User

Write and test WordCount-Python implementation, wordcountpython

Address of this project on github: https://github.com/MaximumRescue/WordCount-Project

Preface

First, I must emphasize that,If you think there is a problem, you must have not carefully read the following instructions.

Let's talk a little bit about the problem. from last week's task allocation to the present, the demand is constantly changing, and the description is also very vague. Since this is only a week's "small job", there is no need to blur the demand. It should be clear about any point, and the samples must be standardized and correct, moreover, if this course focuses on Software Quality Improvement and testing, it should not be vague in terms of demand, which is very unfavorable for everyone's development. This still causes great problems for development.

PSP table

PSP2.1

PSPPhase

Estimated time consumption

(Minutes)

Actual time consumed

(Minutes)

Planning

Plan

   

· Estimate

· Estimate the time required for this task

15 15

Development

Development

   

· Analysis

· Demand analysis (including learning new technologies)

60 80

· Design Spec

· Generate design documents

- -

· Design Review

· Design Review (review design documents with colleagues)

- -

· Coding Standard

· Code specifications (develop appropriate specifications for current development)

- -

· Design

· Specific design

60 90

· Coding

· Specific Encoding

500 450

· Code Review

· Code review

60 40

· Test

· Testing (self-testing, code modification, and modification submission)

120 120

Reporting

Report

   

· Test Report

· Test report

120 140

· Size Measurement

· Computing workload

20 15

· Postmortem & Process Improvement Plan

· Summarize afterwards and propose a process improvement plan

30 30
 

Total

985 980
Design Ideas

1. First, only four basic functions are taken into account: number of characters, number of words, and number of lines. The statistical results are output to the default file in the specified format.

The first thing that comes to mind is modularity. We will construct independent methods for the above different functions. I personally disagree with method nesting, because many times it will lead to confusion, and once the core function is faulty, all other modules will crash.

So here we start to use the python file processing function and the returned file handle method for processing.

(1) The statistical character function is very simple. read () returns a string from start to end of a file, including blank characters such as \ n and \ t. Therefore, you only need to count the length of the returned string.

Note: \ n and \ t are regarded as only one character.

1 # return the character number of a file2 def char_num (): 3 # open the file with the name 'filename' 4 f = open (filename, 'R ') 5 totalstr = f. read () 6 f. close () 7 return filename + ', number of characters:' + str (len (totalstr ))

(2) The row count function is followed. Another method, f. readlines (), returns a list composed of all rows in a file. You only need to count the number of elements in the list.

1 # return the line number of a file2 def line_num (): 3 # open the file with the name 'filename' 4 f = open (filename, 'R ') 5 lines = f. readlines () 6 f. close () 7 return filename + ', number of rows:' + str (len (lines ))

(3) counting the number of words is a little complicated, but it is also very simple. We can search for each row returned above. The search criteria are regular expressions:A word is composed of at least two letters and does not contain any other symbols (not necessarily a. a connector such as five-year-old, abbreviated as don't). You can use re. findall (R' [A-Za-z] {2,} ') to filter all words.

1 # return the word number of a file 2 def word_num (): 3 # open the file with the name 'filename' 4 f = open (filename, 'R ') 5 lines = f. readlines () 6 f. close () 7 # get the word by regex expression 8 linewords = [] 9 for I in range (0, len (lines): 10 linewords. append (re. findall (R' [A-Za-z] {2,} ', lines [I]) 11 # count the number of words12 count = 013 for lineword in linewords: 14 for word in lineword: 15 If word. isalpha () = False or len (word) = 1:16 lineword. remove (word) 17 if lineword! = ['']: 18 count + = len (lineword) 19 return filename + ', number of words:' + str (count)

(4) write the file using the f. write () method. Note that the mode must be a (append) when opening the file. Otherwise, all the previous records will be erased.

1 # write the information to the file2 def write_file(filename, info):3     f = open(filename, 'a')4     f.write(info)5     f.close()

The implementation of these parts is relatively simple. But obviously these methods share one parameter, so for convenience, I designed a class that contains all the methods. For details, refer to the above link.

2. The following describes the extended functions.

Extended functions: recursively process files in the directory that meet the conditions, return code lines, empty lines, and comment lines, and ignore words in the disabled Word Table (Reserved Words ).

(1) starting from the simplest reserved word processing, this is similar to the previous word processing. The difference is that we need to remove the reserved word, so we only need to constructReserved word list, The words obtained by the judgment are not in the table. If the words are deleted, the number of words retained can be counted. Therefore, we can rewrite the previous method. The method for obtaining reserved words is similar to the process for obtaining words:

 1 # get the preserved words list 2 def get_prelist(fname): 3     filename = join(sys.path[0], fname).replace('\\', '\\\\') 4     fhandle = open(filename, 'r') 5     wholestr = fhandle.read() 6     prelist = wholestr.split(' ') 7     for word in prelist: 8         if word == "" or word == "\n" or word == "\t": 9             prelist.remove(word)10     return prelist

(2) Next we will describe the precise statistics of rows. As we have mentioned above, f. readlines () returns a list composed of strings in each row, so we can perform some processing on each string:

Use the str. replace (a, B) method to remove all linefeeds, tabs, and spaces from the string, and then make a judgment:

  If it starts with '//', it is used as the comment line, without considering multi-line comments. If one line has only one character or no character, it is used as a blank line; otherwise, it is regarded as a line of code.

1 # return line details of a file 2 def line_detail (): 3 # open the file with the name 'filename' 4 f = open (filename, 'R ', encoding = 'utf-8') 5 # get all line string into a list 6 lines = f. readlines () 7 f. close () 8 # distinguish different lines 9 codelines, emptylines, commentlines = [], [], [] 10 for line in lines: 11 tmpline = line. replace ('','') 12 tmpline = tmpline. replace ('\ t', '') 13 tmpline = tmpline. replace ('\ n', '') 14 if len (tmpline) = 1 or len (tmpline) = 0: 15 emptylines. append (line) 16 elif tmpline. startswith ('//'): 17 commentlines. append (line) 18 else: 19 codelines. append (line) 20 return filename + ', code line/blank line/comment line:' + str (len (codelines )) + '/' \ 21 + str (len (emptylines) + '/' + str (len (commentlines ))

(3) For nested processing of all eligible files in the folder, we divide the problem:

First, we need to obtain the file type. The idea is very simple. We can obtain the corresponding parameters in the command line, and obtain the corresponding file type by intercepting the end of the string.

Second, we perform recursive search in the entire execution folder Based on the obtained file type to return the obtained FileAbsolute pathList.

 1 # determine the type needed 2 tmplist = desfile.split('.') 3 type = '.' + tmplist[-1] 4 # get the required list 5 localinfo = directory.build_infolist(type) 6 # build up the information list from the file list 7 def build_infolist(type=''): 8     filenames = build_filelist(type) 9     dirnames = build_dirlist()10     # get information by creating a FileInfo object11     infos = []12     for fname in filenames:13         info = FileInfo(path, fname)14         infos.append(info)15     # deal with the inner directory16     for dirname in dirnames:17         new_path = join(path, dirname)18         newdir = DirInfo(new_path)19         new_infos = newdir.build_infolist(type)20         infos = infos + new_infos21     return infos

3. Advanced functions: displays the graphical interface. You can select a single file on the interface, and the program displays related statistics.

It is also the sub-governance. Part of it is to obtain the file path and file name through the displayed dialog box, and the other part is to perform the above operations on the obtained information.

The difficulty lies in the display dialog box. The wx library is used to display the dialog box. The FileDialog method in the dialog box is used to create the dialog box. FileDialog. getdirectory () and FileDialog. getname () are used to obtain the file path and file name. Sample Code:

Note: Only the number of characters, number of lines, and number of words (no reserved words) are displayed)

 1 # A method to get filename by dialog 2 def dialog_get(): 3     app = wx.App() 4     frame = wx.Frame(None) 5     openwildcard = "All files(*.*)|*.*|" + \ 6                     "C/C++ files(*.c;*.cpp;*.h)|*.c;*cpp;*.h|" + \ 7                     "Java source files(*.java)|*.java|" + \ 8                     "Python source files(*.py)|*.py|" + \ 9                     "Text files(*.txt)|*.txt"10     # Create open file dialog11     openFileDialog = wx.FileDialog(frame, "Choose a file to open",12                                    wildcard=openwildcard, 13                                    style=wx.FD_OPEN | wx.FD_FILE_MUST_EXIST)14     openFileDialog.ShowModal()15     dir = openFileDialog.GetDirectory()16     filename = openFileDialog.GetFilename()17     openFileDialog.Destroy()18     return dir, filename
Source code module description
FileInfo. py Class FileInfo

Attributes: path, fname, filename

Methods: char_num (), word_num (prelist ),

Line_num (), line_detail (), write_info ()

DirInfo. py Class DirInfo

Attributes: path

Methods: build_filelist (tyoe), build_dirlist (), build_infolist (type)

ExtraOpt. py - Methods: get_prelist (fname), dialog_get ()
Main. py - (No methods, but some important codes used as main () in java)

In fact, there is also a setup. py file used to package the script file into an exe file, but this is not the focus of our implementation project, so it is not listed.

Software operation instructions

1. Important Points for running the. exe file:

First, describe how to package a python script file into an exe file. Install the py2exe library in advance and construct a setup. py file for packaging. Run the following command for this file:

python <filename>.py py2exe

The setup. py file organization can be like this:

1 from distutils.core import setup2 import py2exe3 setup(console=["main.py"])

So what is the packaging? During the packaging process, create a dist folder, copy some files in the PythonXX original folder to the folder, and then construct some required registries. Shows the approximate file structure:

Therefore, if you need to run it, you must put all the files in a folder with the content in the BIN folder. Its other operations are similar to those in the. exe file.

2. If the .exe file cannot run, you can download the. pyc and main. py files in the _ pycache _ folder, put them in a folder, and run them using python commands.

Python version 3.4 or later, run the Code:

python main.py [arguments]

3. If none of the above works, ensure that Python 3.4 + is installed, install the wxPython library, download all. py files, and run the main. py script. The other content is the same as that in 2.

Test Process Design

1. Explain the preliminary evaluation of the Code after writing:

(1) The structure of the command line parameter processing is not optimized enough, and this part is helpless. The dictionary constructor I use is to associate the file with the corresponding operation parameters, but is this the optimal method, I still have doubts. For this, you need to refer to the parameter Processing Method of the DOS or python terminal;

(2) Here we consider a simple situation where all files can be opened and no exception handling mechanism is added. However, I think exception handling is necessary and has not been improved with limited time.

(3) There seems to be some room for improvement in file deployment and modular design.

In short, I think we still need to make some optimizations here. We still need to learn about python debugger and other tools.

2. Specific test: limited space. Three documents and some results are displayed:

(1) graphic interface test: command:

wc.exe -x

File_1.c

1 {1}2 2

File_2.c

1 a2 a/3 a+b

File_3.c

1 ---2 

The displayed interface is as follows:

The output result is as follows:

1. 
File_1.c, number of characters: 5file_1.c, number of rows: 2file_1.c, number of words: 02.
File_2.c, number of characters: 8file_2.c, number of lines: 3file_2.c, number of words: 03.
File_3.c, number of characters: 4file_3.c, number of rows: 1file_3.c, number of words: 0

(2) single file command test:

Testfile. c

1 {2 }//3 //4              a5 aasa//6 while and or if endif

StopList.txt

while if for

Then test each command:

4.Wc.exe-c testfile. ctestfile. c, number of characters: 525.Wc.exe-w testfile. ctestfile. c, number of words: 66.Wc.exe-l testfile. ctestfile. c, number of rows: 67.Wc.exe-a testfile. ctestfile. c, code line/empty line/comment line: 3/2/18.Wc.exe-w testfile. c-e StopList.txt testfile. c, number of words: 49.Wc.exe-s-c-w-l *. c-e StopList.txt file_1.c, number of characters: bytes, number of words: 0file_1.c, number of lines: 2file_2.c, number of characters: 8file_2.c, number of words: 0file_2.c, number of lines: 3file_3.c, number of words: 0file_3.c, number of rows: 1testfile. c, number of characters: 52testfile. c. Number of words: 4testfile. c. Number of rows: 610.Wc.exe-c-w-l-a testfile. c-o result.txt testfile. c, number of characters: 52testfile. c. Number of words: 6testfile. c. Number of rows: 6testfile. c. code line/empty line/comment line: 3/2/1

The correct results are displayed here. Several problems have occurred and are corrected:

1) the file cannot be read during the-e process. cause: the get_prelist () path is incorrect. After being packaged as exe, the path needs to be redirected again.

Solution: Put cur_file_dir () in ExtraOpt. py and use this method to re-tune the path.

2)-w-e duplicate output: cause: the two settings are in different judgment conditions and are not judged at the same time.

Solution: Put the reserved word list in the-w processing process, and-e obtains the list separately.

There are no other problems, but you mustMake sure that the command line is correctly used.The command line is incorrect.

Summary

The content of this project is quite full. Everyone is graduating soon, and there are also a lot of courses. experienced people know that building a project is no matter how small it is, intentionally or unintentionally, we will try to ensure a relatively clear and reasonable architecture. The code written by many powerful students is very clearly divided by modules, well-structured, and highly robust and reusable. This often means a higher workload. There are still some difficulties in searching information. In a search engine, Baidu belongs to the lowest level, Bing belongs to the intermediate level, and google is the advanced level (although not available). The search results often cannot meet the requirements, A large number of searches and manual filtering are required. Many blogs on professional websites such as CSDN, blog Park, and open-source China are copied and reproduced, and the updates are very slow (maybe now everyone is working on frameworks or AI, no one cares about basic content updates). Many of the features of the language used have been eliminated. The descriptions and examples of new features in official documents are not so well understood (manual documents, is not a textbook ). It often appears that you want to find a function implementation method, and most of the jumps are junk information or useless content, it can be said that a large part of the development process is wasted on the process of extracting useful information.

However, the whole process is worthwhile. However, although I have not detected any problems in my tests, I still need to perform further tests. More complex test cases are necessary. If there is any problem with the above design process or implementation method, I hope you will not be enlightened.

 

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.