How does a python function local variable execute? An analysis of the application of Python function variables

What this article brings to you is about how the Python function local variables are executed. On the application of Python function variables, there is a certain reference value, the need for friends can refer to, I hope to help you.

Objective

These two days in Codereview, when you see this code

# pseudo Code Import Somelibclass A (object):    def load_project (self):        self.project_code_to_name = {} for        project in Somelib.get_all_projects ():            self.project_code_to_name[project] = Project        ...

The intent is simple, which is to somelib.get_all_projects plug the acquired items into theself.project_code_to_name

However, the impression of this is that there is a space for optimization, so proposed adjustment program:

Import Somelibclass A (object):    def load_project (self):        project_code_to_name = {} for        project in Somelib.get_all_projects ():            project_code_to_name[project] = Project        self.project_code_to_name = project_ Code_to_name ...        

The scheme is simple, it is to define the local variables project_code_to_name , the operation is finished, and then assign value to self.project_code_to_name .

In the back of the test, it is also true to find that this will be better, then the results know, the next must be to explore the reason!

Local variables

In fact, many places on the internet, and even a lot of books have spoken of a point of view: access to local variable speed is much faster , rough look like good, and then see the following posted a lot of test data, although do not know what is, but this is the real dick, remember to say, tube him!

But in fact, this point of view has some limitations, not all the same. So let's get to the bottom of this sentence and why everyone likes to say so.

First look at the section code to understand what the local variables are:

#coding: utf8a = 1def Test (b):    c = ' test '        print a   # global variable    print b   # local variable    print c   # local variable test (3)

# Output 13test

In simple terms, a local variable is used only for the function domain in which it is located, and is recycled beyond the scope

If you understand what a local variable is, you need to talk about the love and hate of Python functions and local variables, because it's hard to feel where it's going to be if you don't figure it out.

To avoid boredom, use the above code to illustrate, by the way, the analysis of the dis performed by the test function:

# call_function  5           0 load_const               1 (' Test ')              3 store_fast               1 (c)  6           6 load_global              0 (a)              9 Print_item             print_newline  7          load_fast                0 (b)             Print_item             print_newline  8          load_fast                1 (c)             print_item             print_newline             load_const               0 (None)             Return_value

In the relatively clear can see a, b, c corresponding instruction block, each block of the first line is, as the LOAD_XXX name implies, is to indicate where these variables are obtained from.

LOAD_GLOBALThere's no doubt about it, but LOAD_FAST what is it? Sounds like you should scream LOAD_LOCAL ?

However, the fact is so magical, people are really called LOAD_FAST , because the local variables are read from a called fastlocals array, so the name is called (I guess).

So the main character comes, we have to focus on this, because this is really very interesting.

Python function execution

The construction and operation of Python functions, said the complexity is not complex, said simple is not simple, because it needs to distinguish a lot of situations, such as the need to distinguish between functions and methods, and then the distinction is whether there are parameters, what parameters, there is a variable length of wood parameters, Wood has a key parameter.

It's impossible to say it all, but you can simply plot the approximate process (ignoring the parameter change details):

All the way down the river, direct fast_function , it's called here:

CEVAL.C-Call_functionx = Fast_function (func, Pp_stack, N, Na, NK);

Parameters explained below:

1. Func: Passed in test ;

2. Pp_stack: Approximate Understanding Call stack (py mode);

3. NA: The number of positional parameters;

4. NK: the number of keywords;

5. n = Na + 2 * NK;

So the next step is fast_function to see what to do.

Initializes a wave of

1. Define the CO to hold the test object inside thefunc_code

2. Define globals to hold func_globals (dictionary) inside the test object

3. Define argdefs to hold the test object func_defaults (the default value of the keyword parameter when constructing the function)

Let's make a judgment, if argdefs 为空 && 传入的位置参数个数 == 函数定义时候的位置形参个数 &&没有传入关键字参数

Then it

1. Use 当前线程状态 , co globals to create a new stack object f ;

2. Definition fastlocals (fastlocals = f->f_localsplus;);

3. Plug in all the incoming parameters.fastlocals

So the question is, how to plug it? How to find out what the Ghost parameter: This problem can only be dis answered:

We know now that this step is in the CALL_FUNCTION inside, so the action of the plug parameter is definitely before this, so:

          load_name                2 (test)             load_const               4 (3)             call_function            1             pop_top             Notoginseng load_ CONST               1 (None)             Return_value

On the CALL_FUNCTION above to see 30 LOAD_CONST 4 (3) , interested in children's shoes can try to pass a few parameters, you will find the parameters passed in order through LOAD_CONST such a way to load in, so how to find the parameters of the problem becomes apparent;

fast_function function fastlocals = F->f_localsplus;stack = (*pp_stack)-N; for (i = 0; i < n; i++) {     py_incref (*stack);     Fastlocals[i] = *stack++; }

The presence of n here still remembers how it came about? Look n = na + 2 * nk; at the one above, can you think of anything?

In fact, this place is simple by pp_stack locating the offset n bytes at the beginning of the plug-in parameter.

So the question is, if n is the number of positional arguments + keyword argument, then what does 2 * NK mean? In fact, the answer is very simple, that is, the keyword parameter bytecode is a byte code with parameters, is accounted for 2 bytes.

Here, the Stack object f f_localsplus also boarded the historical stage, but at this time it, but also just a juvenile without personnel, but also need to experience.

Do these actions, and finally come to the real execution of the function of the place: PyEval_EvalFrameEx , here, need to confess, there is a and PyEval_EvalFrameEx very similar, called PyEval_EvalCodeEx , although look like, but people do more work.

Please look back at the fast_function beginning of the first there will be a judgment, we said above is the judgment, that is, the simplest function of the implementation of the situation. If a function passes in a keyword parameter or something else, it's a lot more complicated, and then it needs to be PyEval_EvalCodeEx handled by a wave and then executed PyEval_EvalFrameEx .

PyEval_EvalFrameExThe main work is to parse the bytecode, like just those CALL_FUNCTION , and LOAD_FAST so on, it is parsed and processed by it, its essence is a dead loop, and then there is a heap swith - case , which is basically the nature of Python's operation.

F_localsplus Deposit and Fetch

Speaking of such a long heap, the most basic function of Python is to simply sweep the process of blind, and now began to explore the topic.

For the sake of simple elaboration, direct reference to nouns: fastlocalsfastlocals = f->f_localsplus

Just now simply see, Python will pass in the parameters, in order to plug into the fastlocals inside, then undoubtedly, the location parameter passed, it must belong to local variables, then the keyword parameters? That must also be local variables, because they are treated in a special way.

So in addition to the function parameters, there must be a function inside the assignment? This bytecode was also given earlier in the morning:

# call_function  5           0 load_const               1 (' Test ')              3 store_fast               1 (c)

Here comes the new bytecode STORE_FAST , together with a look at the implementation:

# Pyeval_evalframeex One of the large switch-case branches:        predicted_with_arg (store_fast);        TARGET (store_fast)        {            v = POP ();            SETLOCAL (Oparg, v);            Fast_dispatch ();        } # because there is a macro involved, just by the way: #define GETLOCAL (i)     (Fastlocals[i]) #define SETLOCAL (I, value) do      {pyobject *tmp = getlocal (i ); \                                     getlocal (i) = value; \                                     py_xdecref (TMP);} while (0)

The simple explanation is that the value v obtained by the POP () is plugged into the oparg position of the fastlocals. Here, V is "test" and Oparg is 1. The diagram shows that:

There are children's shoes may suddenly become ignorant, why suddenly come to a b ? We need to go back and see how the test function is defined:

I feel the probability of looking back is very low, give it straight. def test (b):    c = ' Test '        print b   # local variable    print c   # local variable

See the function definition should know, because b it is a parameter ah, old already stuffed in ~

That storage knows, then how to take it? It is also the bytecode of this code:

Load_fast                1 (c)

Although this is all about knowing the principle of the toe, it is fair to give the code:

# Pyeval_evalframeex One of the large switch-case branches: TARGET (load_fast) {    x = getlocal (oparg);    if (x = NULL) {        py_incref (x);        PUSH (x);        Fast_dispatch ();    }    Format_exc_check_arg (Pyexc_unboundlocalerror,        unboundlocal_error_msg,        Pytuple_getitem (co->co_ Varnames, Oparg));    break;}

Directly using GETLOCAL the index in the array to take the value.

When we get here, we should be able to f_localsplus understand the words. This place is not difficult, in fact, generally will not be mentioned to this, because generally ignored can be, but if you want to pay attention to the performance, then this little knowledge should not be ignored.

Variable use posture

Because it is object-oriented, so we are accustomed to the class way through, for the following ways of use, is also handy to come:

Class SS (object):    def __init__ (self):        Self.fuck = {}    def test (self):        print Self.fuck

This approach is generally not a problem, but also very normative. By then, if it is the following operation, then there is a problem:

Class SS (object):    def __init__ (self):        Self.fuck = {}    def test (self):        num = Ten for        i in range (num): C5/>self.fuck[i] = i

The performance loss of this code increases with the value of NUM, and if the following loops involve more read, modify, and more of the class properties, the effect is even greater.

If this class attribute is changed to a global variable, there will be a similar problem, except that the action class property is much more frequent than manipulating the global variable.

Let's look directly at the difference between the two.

Import Timeitclass SS (object):    def Test (self):        num =        Self.fuck = {}        # in order to be fair, each execution will also initialize the new {} for        I in range (num):            self.fuck[i] = i    def test_local (self):        num =        fuck = {}             # for fairness, each execution initializes the new {} for        I in range (num):            fuck[i] = i        self.fuck = fucks = SS () print Timeit.timeit (stmt=s.test_local) Print Ti Meit.timeit (Stmt=s.test)

As you can see, the greater the value of NUM, the more times the For loop will be, and the greater the gap between the two.

Then why is this, also in the bytecode can be seen writing clues:

//s.test >> for_iter (to) STOR             E_fast 2 (i) 8 Load_fast 2 (i) PNS Load_fast 0 (self)              Load_attr 0 (hehe) load_fast 2 (i) STORE_SUBSCR                Jump_absolute >> pop_block//s.test_local >> for_iter             (To) Store_fast 3 (i) Load_fast 3 (i)             Load_fast 2 (hehe) Notoginseng Load_fast 3 (i) + STORE_SUBSCR             Jump_absolute >> pop_block load_fast 2 (hehe) Load_fast 0 (self) Wuyi store_attr 1 (hehe) 

The above two paragraphs is the content of two methods for block , we will know the comparison, s.test compared to s.test_local , more than one LOAD_ATTR put in FOR_ITER and POP_BLOCK between.

What does that mean? This shows that in each cycle, it is s.test necessary LOAD_ATTR , naturally, that we need to see what this is about:

TARGET (load_attr) {     w = GETITEM (names, oparg);     v = TOP ();     x = Pyobject_getattr (V, W);     Py_decref (v);     Set_top (x);     if (x = NULL) DISPATCH ();     Break }# related macros Define # define GETITEM (V, i) Pytuple_getitem ((v), (i))

Here comes a strange variable name , what is this? In fact, this is an array of names maintained by each codeobject, and basically the string used by each block will be stored here, as well as orderly:

Pycodeobject structure member Pyobject *co_names;        /* List of strings (names used) */

Then LOAD_ATTR the task is clear: first remove the string from the name list, the result is "hehe", and then through the pyobject_getattr to find, here is in the S instance to find.

And do not say the search efficiency how, light more this step, can near misses the difference of thousands of miles, of course, this is in the case of more frequent operation times.

So we will be in some 类/实例属性 cases of frequent operation, should be taken out of the first to 属性 save 局部变量 , and then use 局部变量 to complete the operation. Finally, the changes are updated as appropriate 属性 .

At last

In fact, compared to variables, the use of functions and methods are more learned, more worthy of exploration, because that principle and the surface looks more different, the next opportunity to explore. Usually work more attention, in order to make our PY can be a little bit faster point point.