Python source code analysis notes: Dict objects in python

Source: Internet
Author: User

This article is senlie original, reproduced Please retain this address: http://blog.csdn.net/zhengsenlie


1. PyDictObject object --> map in C ++ STL is based on RB-tree, and the search time complexity is O (logN)
PyDictObject uses a hash table. The time complexity is O (1)

Typedef struct {Py_ssize_t me_hash; // hash value of me_key to avoid re-calculating the hash value PyObject * me_key; PyObject * me_value;} PyDictEntry each query;

The (key, value) pair is called an entry, which can be converted between three states:
Unused state --> both me_key and me_value are NULL
Active State --> neither me_key nor me_value is NULL
Dummy state --> me_key is dummy, me_value is NULL
Figure 5-2
Typedef struct _ dictobject PyDictObject; struct _ dictobject {incluma_fill; // number of elements: Active + DummyPy_ssize_t ma_used; // number of elements: ActivePy_ssize_t ma_mask; // record the number of entries PyDictEntry * ma_table; PyDictEntry * (* ma_lookup) (PyDictObject * mp, PyObject * key, long hash); PyDictEntry ma_smalltable [PyDict_MINSIZE];}

PyDict_MINSIZE is set to 8 by default.
When the number of entries of A PyDictObject object is less than 8, ma_table points to ma_smalltable
When the number of entries of A PyDictObject object is greater than 8, ma_table points to the requested memory space.
Q: What should I do with the objects in ma_smalltable at this time?
A: All objects in ma_smalltable are copied to the new table.
Figure 5-3

PyDict_Type --> PyDictObject type object
PyTypeObject PyDict_Type = {    PyVarObject_HEAD_INIT(&PyType_Type, 0)    "dict",    sizeof(PyDictObject),    0,    (destructor)dict_dealloc,                   /* tp_dealloc */    (printfunc)dict_print,                      /* tp_print */    0,                                          /* tp_getattr */    0,                                          /* tp_setattr */    (cmpfunc)dict_compare,                      /* tp_compare */    (reprfunc)dict_repr,                        /* tp_repr */    0,                                          /* tp_as_number */    &dict_as_sequence,                          /* tp_as_sequence */    &dict_as_mapping,                           /* tp_as_mapping */    (hashfunc)PyObject_HashNotImplemented,      /* tp_hash */    //...};

2. Create PyDictObject
One way:
PyDict_New

PyObject * PyDict_New (void) {register dictobject * mp; // [1]: automatically creates a dummy object // prevents detection sequence interruption if (dummy = NULL) {/* Auto-initialize dummy */dummy = PyString_FromString ("
 
  
");} If (num_free_dicts ){...... // [2]: Use buffer pool} else {// [3]: Create a PyDictObject object mp = PyObject_GC_New (dictobject, & PyDict_Type); EMPTY_TO_MINSIZE (mp );} mp-> ma_lookup = lookdict_string; return (PyObject *) mp;} // EMPTY_TO_MINSIZE --> ma_size, ma_fill = 0 // INIT_NONZERO_DICT_SLOT --> direct ma_table to ma_smalltable
 

Element search
Static PyDictEntry * lookdict (PyDictObject * mp, PyObject * key, register long hash) {register size_t I; register size_t perturb; register PyDictEntry * freeslot; // freeslot is used to point to the first entry register size_t mask = (size_t) mp-> ma_mask in the probe sequence; PyDictEntry * ep0 = mp-> ma_table; register PyDictEntry * ep; register int cmp; PyObject * startkey; // [1]: hash, which locates the first entry I = (size_t) hash & mask of the Conflict Detection chain; // map the hash value to the hash table Ep = & ep0 [I]; // [2]: // 1. the entry is in the Unused State // 2. the key in the entry matches the key to be searched if (ep-> me_key = NULL | ep-> me_key = key) // ** reference the same check return ep; // [3]: The first entry is in the Dummy state. Set freeslot if (ep-> me_key = dummy) freeslot = ep; else {// check the Active state entry if (ep-> me_hash = hash) {startkey = ep-> me_key; cmp = PyObject_RichCompareBool (startkey, key, Py_EQ ); // ** Same Value Check if (cmp <0) return NULL;} freeslot = NULL;} (Perturb = hash; perturb >>= PERTURB_SHIFT) {// [5]: Find the next entry I = (I <2) + I + perturb + 1 on the probe chain; //? Ep = & ep0 [I & mask]; // [6]: the entry in the Unused state is reached, and the search fails if (ep-> me_key = NULL) // If freeslot is not empty, return the entry indicated by freeslot // If freeslot is empty, return the entry return freeslot In the Unused state = NULL? Ep: freeslot; if (ep-> me_key = key) // check return ep by referencing the same rule; if (ep-> me_hash = hash & ep-> me_key! = Dummy) {// "Same Value" Rule Check startkey = ep-> me_key; cmp = PyObject_RichCompareBool (startkey, key, Py_EQ); if (cmp> 0) return ep ;} else {return lookdict (mp, key, hash) ;}} else if (ep-> me_key = dummy & freeslot = NULL) freeslot = ep ;}} // If the entry fails or succeeds, the me_value of the entry is NULL.

Insert and delete
Insertdict
1. If the search is successful, an entry in the Active state is returned. Replace me_value directly.
2. If the search fails, the entry in the Unused or Dummy state is returned. Complete settings include me_key, me_hash, and me_value.
Before insertdict is called, PyDict_SetItem is called.
1. Calculate the hash value
2. insert (key, value) element pair // call insertdict here
3. Adjust the dict memory space when necessary // adjust the dict memory space when the search fails and the loading rate is greater than or equal to 2/3
Call dictresize to change the table size of dict
1. determine the size of the new table
2. If the new table size is 8, you can directly use ma_smalltable
3. Otherwise, you need to apply for space on the stack.
4. Set a new table
5. process the entry in the old table:
Active entry, moved to the new table
In Dummy state, adjust the reference count of the key and discard the entry.
6. Release the memory space maintained by the old table if necessary
PyDict_DelItem
1. Obtain the hash value
2. Search for entry
3. Delete the elements maintained by the entry and convert the entry state to dummy state.

3. PyDictObject Object Buffer Pool
Like the buffer pool used in PyListObject, there is nothing at first. When the first PyDictObject is destroyed,
This buffer pool begins to accept the buffered PyDictObject object.
Static voiddict_dealloc (register PyDictObject * mp) {register PyDictEntry * ep; Py_ssize_t fill = mp-> ma_fill; // [1]: adjust the reference count of objects in dict for (ep = mp-> ma_table; fill> 0; ep ++) {if (ep-> me_key) {-- fill; py_DECREF (ep-> me_key); Py_XDECREF (ep-> me_value) ;}// [2]: Release the memory space applied from the System Heap if (mp-> ma_table! = Mp-> ma_smalltable) PyMem_DEL (mp-> ma_table); // [3]: Put the destroyed PyDictObject object into the buffer pool if (numfree <PyDict_MAXFREELIST & Py_TYPE (mp) ==& PyDict_Type) free_list [numfree ++] = mp; else Py_TYPE (mp)-> tp_free (PyObject *) mp); Py_TRASHCAN_SAFE_END (mp )}

When you create a new PyDictObject object, if there are available objects in the buffer pool, you can directly retrieve and use them from the buffer pool without re-creating them.




Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.