Implementation of list in Python

Original link
This article describes how the list is implemented in Python.
List is particularly useful in python. Let's look at how the list is implemented inside.
Take a look at the following simple program, add some integers to the list and print them out.

>>> L = []>>> L.append(1)>>> L.append(2)>>> L.append(3)>>> L[1, 2, 3]>>> for e in L:... print e... 1 2 3 

As you can see, the list is iterative.

The C structure of the list object

The list in Python is represented by the structure of the C language below. ob_itemis an array of pointers used to hold elements, allocated is the ob_item total amount of memory pre-allocated

typedef struct {    PyObject_VAR_HEAD    PyObject **ob_item;    Py_ssize_t allocated;} PyListObject;

Initialization of the list

Let's take a look at what happens when you initialize an empty list L = []

Arguments:size of the list =0Returnslist object = []pylistnew:nbytes = size * size of global Python object = 0 allocate new list object allocate list of pointers (ob_item) of size Nbytes = 0 clear ob_item set list ' s allocated var to 0 = 0 Slots Return List object                    

It is very important to know the size of the list application memory space (the latter is replaced by allocated) and the size of the space that the list actually stores the elements ( ob_size ), and the size ob_size is the len(L) same as the The size of the allocated is the amount of space that has been applied in memory. Usually you will see that the value of the allocated is ob_size greater than the value. This is to avoid calling realloc for memory allocations each time a new element is added to the list. And then we'll see more about that.

Append

We append an integer to the list: L.append (1). What happened? Called the internal C function App1 ()

arguments: list object, new elementreturns: 0 if OK,-1 if notapp1:n = size of list call list_resize () to resize the list to size n+1 = 0 + 1 = 1 list[n] = list[0] = new element return 0              

Let's take a look at list_resize () , list_resize () will request extra space to avoid calling multiple list_resize () functions, List growth models are: 0, 4, 8, +, +, (+), .....

arguments: list object, new sizereturns: 0 if OK,-1 if notlist_resize:new_allocated = (newsize >> 3" + (NewSize < 9? 3: 6) = 3 new_allocated + = NewSize = Span class= "Hljs-number" >3 + 1 = 4  Resize ob_item  (list of pointers) to Size new_allocated return 0             

Opens up four memory spaces to hold the elements in the list, and the first element to hold is 1. You can see l[0] pointing to the element we just added. A dashed box represents a memory space that has been requested but not yet used (storage element)

We continue to add an element: L.append (2). Called list_resize , while n+1=2. But because allocated (translator Note: The size of the space already applied) is 4. So there's no need to apply for new memory space. The same thing happens when you add two elements to a list again: L.append (3), L.append (4). Shows what we have done so far.

Insert

Now we insert an integer 5:l.insert (1, 5) in the first position of the list to see what's going on inside. INS1 () was called

Arguments:list object, where, new elementreturns:0If OK,-1IfNOTINS1:ResizeListto size n+ 1 = 5, 4 more slots'll be Allocated starting at the last Span class= "Hljs-keyword" >element up to the offset where, right shift each element set new element at offset  where return 0        

The dashed box indicates that memory has been applied but not used. 8 memory space is applied but the list is actually used to store the element with only 5 of the memory space
Time complexity of INSERT is O (n)

Pop

When you pop up the last element of the list: L.pop (). Call Listpop (), which list_resize is called inside the function Listpop () if at this point ob_size (the translator note: after the popup element) is less than half of the allocated (translator Note: The memory space that has been applied). The requested memory space will be reduced.

arguments: list objectreturns:element Poppedlistpop: if list Empty:  Return null resize list with size 5- 1 = 4. 4 is isn't less than 8/2 so no shrinkage Span class= "hljs-built_in" >set list object size to 4 return last element            

The time complexity of the pop is O (1)

You can see that number 4th is pointing to the value (the value that the translator is playing out), but it is important that it is ob_size now 4.
Let's pop an element again. list_resizeInternally, size–1 = 4–1 = 3 is smaller than half of the allocated (space already applied). So the list's application space shrinks to
6, the actual use of the list of space is now 3 (translator note: According to (newsize >> 3) + (NewSize < 9? 3:6) = 3 at the end of the article is detailed)
You can see that () numbers 3rd and 4th also store some integers, but the actual use (storage element) of the list has only 3 space.

Remove

The Python list object has a method to remove a specified element. Call Listremove ().

arguments:list  Object, element to removereturns None Span class= "Hljs-keyword" >if ok, null if notlistremove:loop through each list element: if correct Element:slice list between Element ' s slots and Element ' s slot + ' Span class= "Hljs-number" >1 return none return null              

Cut list and delete element, called list_ass_slice() (Translator Note: In the above slice list between element's slot and element ' s slot + 1 is called), see list_ass_slice() how it works. Here, the low is 1 high for 2 (translator Note: Incoming parameters), we remove data stored in the 1th memory space 5

list object, low offset, high offsetreturns: 0 if OKlist_ass_slice:    copy integer 5 to recycle list to dereference it shift elements from slot 2 to slot 1 resize list to 5 slots return 0

The time complexity of Remove is O (n)

Translator Note:

The sort part of list is not translated
Core part

// 8) + (newsize < 9 and 3 or 6)。当 newsize >= allocated,自然按照这个新的长度 "扩容" 内存。而如果 newsize < allocated,且利用率低于一半呢?allocated    newsize       new_size + new_allocated10           4             4 + 320 9 9 + 7很显然,这个新长度小于原来的已分配空间长度,自然会导致 realloc 收缩内存。(不容易啊)

引自《深入Python编程》

Implementation of list in Python