In-depth discussion of the causes of the default values of Python function parameters, and in-depth discussion of python

In-depth discussion of the causes of the default values of Python function parameters, and in-depth discussion of python

This article introduces the potential harm of using mutable objects as default values of Python function parameters, as well as its implementation principle and design purpose.
Trap Reproduction

We will use actual examples to demonstrate the main content we will discuss today.

The following code defines a function named generate_new_list_with. This function is designed to create a list containing the given element value each time it is called. The actual running result is as follows:
 

Python 2.7.9 (default, Dec 19 2014, 06:05:48)[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwinType "help", "copyright", "credits" or "license" for more information.>>> def generate_new_list_with(my_list=[], element=None):...   my_list.append(element)...   return my_list...>>> list_1 = generate_new_list_with(element=1)>>> list_1[1]>>> list_2 = generate_new_list_with(element=2)>>> list_2[1, 2]>>>

The code running result is not as expected. In the second call of the function, list_2 does not obtain a new list and fill in 2. Instead, it appends a 2 value based on the first call result. Why does this happen when the design bug is the same in other programming languages?
Prerequisites: essence of Python Variables

To understand the cause of this problem, we first need a preparation knowledge, that is, how is Python variable implemented?

Unlike the Declaration and value assignment methods of other programming languages, Python variables are implemented by creating and directing pointer-like methods. That is, the variables in Python are actually a pointer to values or objects (simply put, they are worth a name ). Let's look at an example.
 

p = 1p = p+1

For traditional languages, the preceding code execution method is to declare a p variable in the memory, and then store 1 in the memory where the Variable p is located. When the addition operation is executed, the result of 2 is obtained, and the value of 2 is saved to the memory address of p again. It can be seen that the value on the memory address of Variable p is changed throughout the execution process.

In this Code, Python actually creates a 1 object in the execution memory and Points p to it. When the addition operation is executed, a new object of 2 is actually obtained through the addition operation, and p is directed to this new object. It can be seen that during the entire execution process, the change is the memory address pointed to by p.
Root Cause of default value trap of function parameters

One sentence: the default parameter value of a Python function is bound at the compilation stage.

Now, we will analyze the cause of this trap in detail from an excerpt. The following is an explanation of the reason for extracting from Python Common Gotchas:

Python's default arguments are evaluated once when the function is defined, not each time the function is called (like it is in say, Ruby ). this means that if you use a mutable default argument and mutate it, you will and have mutated that object for all future callto the function as well.

It can be seen that the default value of the parameter is determined in the compile phase of function compilation. For all subsequent function calls, if the parameter is not displayed and assigned a value, the default value of the parameter is just a pointer to the object that already exists in the compile stage. If the value of the specified input parameter is not displayed when a function is called. In this case, this parameter will exist as an alias for the object created during compilation.

If the default value of a parameter is an Imuttable value, if this parameter is modified in the function body, the parameter points to another new Imuttable value. However, if the default value of the parameter is a Muttable object like the example in the beginning of this article, the situation will be worse. All changes to this parameter in the function body are actually modifications to the object that has been determined in the compile stage.

This trap is also prompted in the official Python documentation:

Important warning: The default value is evaluated only once. this makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes. for example, the following function accumulates the arguments passed to it on subsequent CALS:
How to avoid this trap brings unnecessary trouble

Of course, the best way is not to use a mutable object as the default value of the function. If so, the following is a solution. Take the requirement starting with an article as an example:
 

def generate_new_list_with(my_list=None, element=None):  if my_list is None:    my_list = []  my_list.append(element)  return my_list

Why is Python designed like this?

The answer to this question can be found in StackOverflow. The most important part of the answer with the most votes is as follows:

Actually, this is not a design flaw, and it is not because of internals, or performance.

It comes simply from the fact that functions in Python are first-class objects, and not only a piece of code.

As soon as you get to think into this way, then it completely makes sense: a function is an object being evaluated on its definition; default parameters are kind of "member data" and therefore their state may change from one call to the other-exactly as in any other object.

In any case, Effbot has a very nice explanation of the reasons for this behavior in Default Parameter Values in Python.

I found it very clear, and I really suggest reading it for a better knowledge of how function objects work.

In this answer, the answerer considers that a function is an internal level-1 object for the implementation of the Python compiler. The default parameter value is the property of this object. In any other language, object attributes are bound when an object is created. Therefore, it is not surprising that the default values of function parameters are bound at compilation.
However, there are also many other respondents who don't buy it. They think that even a first-class object can be bound during execution in closure mode.

This is not a design flaw. it is a design demo-; perhaps a bad one, but not an accident. the state thing is just like any other closure: a closure is not a function, and a function with mutable default argument is not a function.

There is even a counterargument who put aside the implementation logic, simply from the design point of view: as long as it is against the procedural ape's basic thinking logic, it is a design defect! Below are some of their arguments:

> Sorry, but anything considered "The biggest WTF in Python" is most definitely a design flaw. this is a source of bugs for everyone at some point, because no one expects that behavior at first-which means it shoshould not have been designed that way to begin.

The phrases "this is not generally what was intended" and "a way around this is" smell like they're documenting a design flaw.

Well, the answer to this question will always be a mystery if it does not come from Chen Qing, a Python author.