This article and everyone to share is mainly python automation operations and advanced functions related content, come together to see it, I hope to learn python to help you. first, the co-process1.1 Concept of the association Process co-process, also known as micro-threading, fiber. English name coroutine. One sentence explains what a thread is: The process is a lightweight thread that is user-state. (In fact, it does not indicate white ~) So it's easier to understand the process:threads are system-level, they are scheduled by the operating system, and the process is program-level, which is dispatched by the programmer on demand. We call one of the functions in a thread called subroutines, then the subroutine can be interrupted in the execution of the other subroutine, the other subroutine can also be interrupted back to continue to execute the previous subroutine, this is the process. That is, a piece of code under the same thread execution can be interrupted, and then jump to execute another section of code, when again back to execute the code block, and then from the place where the previous break to start execution. The comparative Professional understanding is:The co-process has its own register context and stack. When the schedule is switched, the register context and stack are saved elsewhere, and the previously saved register context and stack are restored when it is cut back. Thus: The process can retain the state of the last invocation (that is, a specific combination of all local states), each time the procedure is re-entered, which is equivalent to the state of the last call, in other words: The position of the logical flow at the last departure. advantages and disadvantages of 1.2 process Advantages of the co-process: (1) without the overhead of thread context switching, the process avoids meaningless scheduling, which can improve performance (but therefore, programmers have to assume responsibility for scheduling, while the co-process also loses the standard thread using multi- CPU 's ability) (2) cost of locking and synchronizing without atomic operation (3) Easy to switch the control flow, simplify the programming model (4) high concurrency + High scalability + low cost: One CPU Supporting tens of thousands of processes is not a problem. Therefore, it is suitable for high concurrency processing. Disadvantages of the co-process: (1) Unable to take advantage of multicore resources: The nature of the process is single-threaded , it cannot simultaneously Use multiple cores of a single CPU , the coprocessor needs to work with the process to run on multiple CPUs . Of course, most of the applications we write on a daily basis are not necessary, unless CPU intensive applications. (2) blocking (Blocking) operation (e.g. IO ) blocks the entire program 2. How to implement the co-process in Python 2.1 yield Implementation co-process The " subroutine (function) can be interrupted to execute other subroutines during execution, and the other subroutine can be interrupted to continue executing the subroutine before ", so it is easy to think with the yieldof Python , It is obvious that yield is capable of achieving this switchover. def Eater (name): print ("%s eat food"%name) while True: food = yieldprint ("Done") G = Eater ("Gangdan") Print (g) Execution Result:<generator object eater at 0x0000000002140fc0> The result of execution can prove that G is now a generator function 2.2 Function Assignment process Use the expression form of yield, run next (), let the function initialize and stop at yield, and then Send () ,send assigns a value to yield when the next execution of the code is triggeredNext () and send () all allow the function to continue running at the last paused position. def creater (name): print ('%s start to eat food '%name) Food_list = [] while True:Food = yield Food_list print ('%s get%s, to start eat '% (Name,food)) Food_list.append (food) # get generator builder = creater (' Tom ') # now is the run function, let the function initialize Next (builder) Print (builder.send (' bun ')) Print (Builder.send (' bones ')) Print (builder.send (' vegetable soup ')) Execution Result:Tom start to eat food Tom get bun, to start eat [' bun Tom get bone, to start eat [' bun ', ' bone Tom get Vegetable soup , to start eat [' bun ', ' bone ', ' vegetable soup ' It is important to note that each time you need to run the next () function so that the program stays in the yield position. If you have more than one of these functions , you need to execute the next () function to keep the program in the yield position. To prevent forgetting to initialize the next operation, you need to use an adorner to solve this problem def Init (func): def wrapper (*args,**kwargs): Builder = func (*args,**kwargs) Next (builder) # This place is the key you can use builder.send ("None"), the first time you must pass in None. return builderreturn [email protected] creater (name): print ('%s start to eat food '%name) Food_list = [] while True:Food = yield Food_list print ('%s get%s, to start eat '% (Name,food)) Food_list.append (food) # get generator builder = creater ("Tom") # is now a direct run function, no more function initialization Print (builder.send (' bun ')) Print (Builder.send (' bones ')) Print (builder.send (' vegetable soup ')) Execution Result:Tom start to eat food Tom get bun, to start eat [' bun Tom get bone, to start eat [' bun ', ' bone Tom get Vegetable soup , to start eat [' bun ', ' bone ', ' vegetable soup ' simple application of 2.3 process function Please feed Tom food def Init (func): def wrapper (*args,**kwargs): Builder = func (*args,**kwargs) Next (builder) return builderreturn [email protected] creater (name): print ('%s start to eat food '%name) Food_list = [] while True:Food = yield Food_list print ('%s get%s, to start eat '% (Name,food)) Food_list.append (food) def food (): builder = creater ("Tom") while True:Food = input (" please feed Tom :"). Strip () if food = = "Q": print ("Feed end ") return 0 Else:builder.send (food) if __name__ = = ' __main__ ': food ()Execution Result:Tom start to eat food Please feed Tom food: Bones Tom get bone, to start eat please give Tom food: Vegetable Soup Tom get Vegetable soup , to start eat Please feed Tom food:q Feeding Endsapplication of 2.4 in the process of association function Implement the "Grep-rl Error < directory >" command in Linux , Filter the contents of a file under the sub-file, Word folder content of the function program First understand the walk method in an OS module to Open the folder under the path in the parameter and return a tuple >>> import OS # import module >>> os.walk (r "E:\Python\script") # using R is to let the symbols in the string have no special meaning, for the escape >>> g = Os.walk (r "E:\Python\script") >>> next (g) (' E:\\python\\script ', ['. Idea ', ' functions '], []) returns a tuple, the first element is the path to the file, the second is a folder, and the third is the file under that paththere is a need to write the idea of a program: process-oriented Programmingsecond, process-oriented programmingProcess- oriented: The core is the process of two words, process and the steps to solve the problem, based on process-oriented design program is an industrial pipeline, is a mechanical way of thinking. Pipeline programming idea, in the design process, need to design the entire processAdvantages:1:architecture clearer 2:simplifying the complexity of the program Disadvantages:Scalability is extremely poor, so the process-oriented scenario is: Software that does not require frequent changes, such as:Linux kernel,httpd,git and other software The following is a process-oriented concept to complete the function of the application of the functiondirectory structure: Test ├──aa │├──bb1 ││└──file2.txt │└──bb2 │└──file3.txt └─file1.txt file content:file1.txt:error123file2.txt:123file3.txt: 123ErrorProgram FlowFirst stage: Find the absolute path to all filesSecond Stage: Open FilePhase three: Iterate through each line Fourth stage: Filtering "Error"Fifth stage: Print the file name that the line belongs toFirst stage: Find the absolute path to all filesG is a generator that can be executed with next () , and each time next is run, The result here is to open the path to the file in turn >>> Import os>>> g = Os.walk (r "E:\Python\script\ function \test") >> > Next (g) (' e:\\python\\script\\ function \\test ', [' AA '], []) >>> next (g) (' e:\\python\\script\\ function \\test\\aa ', [' bb1 ', ' bb2 '], [' file1.txt ']) >>> Next (g) (' e:\\python\\script\\ function \\test\\aa\\bb1 ', [], [' File2.txt ']) >>> next (g) (' e:\\python\\script\\ function \\test\\aa\\bb2 ', [], [' File3.txt ']) >>> next (g) Traceback (most recent): File "File:///c:\users\wlc\appdata\local\temp\ksohtml\wps86f5.tmp.wmf", line 1, in stopiterationwhen we open the file, we need to find the absolute path of the file, and now we can stitch the first and third parts with the method of string concatenation.To Open with a loop: import osDir_g = Os.walk (r "E:\Python\script\ function \test")for Dir_path in Dir_g: Print (Dir_path) Results:(' e:\\python\\script\\ function \\test ', [' AA '], []) (' e:\\python\\script\\ function \\test\\aa ', [' bb1 ', ' bb2 '], [' file1.txt ']) (' e:\\python\\script\\ function \\test\\aa\\bb1 ', [], [' File2.txt ']) (' e:\\python\\script\\ function \\test\\aa\\bb2 ', [], [' File3.txt ']) stitching the queried files and paths into absolute paths import osDir_g = Os.walk (r "E:\Python\script\ function \test")for Dir_path in Dir_g: for file in dir_path[2]: file = "%s\\%s"% (dir_path[0],file) Print (file) Execution Result: E:\Python\script\ function \test\aa\file1.txt E:\Python\script\ function \test\aa\ Bb1\file2.txtE:\Python\script\ function \test\aa\bb2\file3.txt implemented with functions :Import Osdef Search (): while True: dir_name = yielddir_g = Os.walk (dir_name) for Dir_path in Dir_g: for file in dir_path[2]: file = "%s\\%s"% (dir_path[0],file) Print (file) g = search () Next (g) G.send (r "E:\Python\script\ function \test") in order to return the results to the next process@init # Initialize generator def search: while True: dir_name = yielddir_g = Os.walk (dir_name) for Pardir,_,files in Dir_g: for file in Files: Abspath = r "%s\%s"% (pardir,file) Target.send (abspath) Second Stage: Open File@initdef Opener (target): while True: Abspath=yieldwith open (Abspath, ' RB ') as F: Target.send ((abspath,f)) Phase three: Iterate through each line of content@initdef Cat (target): while True:Abspath,f=yield # (abspath,f) For line in F: Res=target.send ((abspath,line)) if Res:breakFourth stage: Filtration@initdef grep (pattern,target): Tag=false while True: Abspath,line=yield tag Tag=falseIf pattern in line: Target.send (abspath) tag=trueFifth stage: Print the file name that the line belongs to@initdef printer (): while True: Abspath=yieldPrint (abspath) g = Search (opener (' Error ' encode (' Utf-8 '), Printer ()))) g.send (R ' E:\Python\script\ function \test ') Execution Result:E:\Python\script\ function \test\aa\file1.txt E:\Python\script\ function \test\aa\bb2\file3.txt
Source: 51CTO
A detailed description of the advanced functions of Python Learning
