Understanding the implementation of the array within PHP (PHP source for PHP Developers-part fourth)

Article from: http://www.aintnot.com/2016/02/15/understanding-phps-internal-array-implementation-ch

Original: https://nikic.github.io/2012/03/28/Understanding-PHPs-internal-array-implementation.html

Welcome to the fourth part of the "PHP Source for PHP developers" series, which we'll talk about how the PHP array is represented internally and used in the code base.

To prevent you from missing the previous article, here is the link:

• The first part: PHP source code for PHP developers-source structure

• Part II: Understanding the definition of PHP intrinsic functions

• Part III: PHP's variable implementation

Everything is a hash table.

Basically, everything in PHP is a hash table. Not just in the following PHP array implementations, they are also used to store object properties, methods, functions, variables, and almost everything.

Because the hash table is too basic for PHP, it's worth delving into how it works.

So, what is a hash table?

Remember, in C, arrays are memory blocks, and you can access them by subscript. Therefore, arrays in C can only use integers and ordered key values (that is, you cannot use the 1332423442 key value after the key value 0). There's no such thing as an associative array in C.

A hash table is something like this: they use a hash function to convert a string key value to a normal integer key value. The result of the hash can be used as the key value of the normal C array (also known as a memory block). The problem now is that there is a conflict in the hash function, which means that multiple string key values may generate the same hash value. For example, in PHP, an array of more than 64 elements, the string "foo" and "Oof" have the same hash value.

This can be done by storing potentially conflicting values in the linked list, rather than storing the values directly in the generated subscript.

Hashtable and Buckets

So, now that the basic concept of a hash table is clear, let's look at the hash table structure implemented within PHP:

typedefstruct_hashtable {UINTntablesize; UINTNtablemask; UINTnnumofelements; ULONGnnextfreeelement; Buckets*Pinternalpointer; Buckets*Plisthead; Buckets*Plisttail; Buckets**arbuckets;    dtor_func_t Pdestructor;    Zend_bool persistent; unsignedCharNapplycount;     Zend_bool bapplyprotection; #ifZend_debugintinconsistent; #endif} HashTable;

To quickly cross:

nNumOfElementsIdentifies the number of values that are now stored in the array. This is also count($array) the value returned by the function.

nTableSizeRepresents the capacity of a hash table. It is usually the power value of the next greater than or equal nNumOfElements to 2. For example, if the array stores 32 elements, then the hash table is also a 32 size capacity. But if one more element is added, that is, the array now has 33 elements, then the hash table's capacity is adjusted to 64.

这是为了保持哈希表在空间和时间上始终有效。很明显，如果哈希表太小，那么将会有很多的冲突，而且性能也会降低。另一方面，如果哈希表太大，那么浪费内存。2的幂值是一个很好的折中方案。

nTableMaskIs the capacity of the hash table minus one. This mask is used to adjust the generated hash value based on the current table size. For example, the true hash value of "foo" (using the djbx33a hash function) is 193491849. If we now have a 64-capacity hash table, we obviously can't use it as an array subscript. Instead, the mask of the hash table is applied, and then only the low of the hash table is taken.

hash           |        193491849 |     0b1011100010000111001110001001& mask         | &             63 | &   0b0000000000000000000000111111---------------------------------------------------------= index        | = 9              | =   0b0000000000000000000000001001

nNextFreeElementIs the next number key value that can be used when you use $array[] = xyz is used.

pInternalPointerStores the current location of the array. This value can be accessed using the Reset (), current (), key (), Next (), Prev (), and end () functions during the foreach traversal.

pListHeadAnd pListTail identifies the position of the first and last elements of the array. Remember: The array of PHP is an ordered set. For example, [' foo ' = ' bar ', ' bar ' = ' + ' foo '] and [' Bar ' = ' foo ', ' foo ' + ' bar '] These two arrays contain the same elements, but in a different order.

arBucketsIs the "hash table (internal C array)" We often talk about. It is defined in bucket * *, so it can be treated as a bucket pointer to an array (we'll talk about what buckets are right away).

pDestructorIs the destructor of the value. If a value is removed from HT, then this function is called. The common destructor is zval_ptr_dtor. Zval_ptr_dtor will reduce the number of references to Zval, and if it encounters O, it will destroy and release it.

The last four variables are not so important to us. So simply to say that the persistent identity hash table can survive in multiple requests, napplycount and bapplyprotection prevent multiple recursion, inconsistent is used to capture the illegal use of hash tables in debug mode.

Let's continue with a second important structure: buckets:

typedefstructBucket {ULONGh; UINTnkeylength; void*PData; void*pdataptr; structBucket *Plistnext; structBucket *Plistlast; structBucket *Pnext; structBucket *PLast; Const Char*Arkey;} Buckets;

his a hash value (no value before the Mask value mapping is applied).

arKeyUsed to hold string key values. nKeyLengthis the corresponding length. If it is a numeric key value, neither of these variables will be used.

pDataAnd pDataPtr is used to store the true value. For a PHP array, its value is a zval struct (but it is also used elsewhere). Don't dwell on why there are two attributes. The difference between them is who is responsible for releasing the value.

pListNextAnd pListLast identifies the next element and the previous element of an array element. If PHP wants to iterate through the array sequentially, it will start with the bucket of Plisthead (inside the hashtable structure) and use the Plistnext bucket as the traversal pointer. The same is true in reverse order, starting with the Plisttail pointer and then using the Plistlast pointer as the variable pointer. (You can call end () in the user code and call the Prev () function to achieve this effect.) ）

pNextand pLast generate the "list of possible conflicting values" that I mentioned above. The Arbucket array stores the buckets of the first possible value. If the bucket does not have the correct key value, PHP will look for the bucket pointed to by Pnext. It will always point back to the bucket until it finds the right bucket. Plast is the same principle in reverse order.

As you can see, PHP's hash table implementation is quite complex. This is the price it will pay to use the super-flexible array type.

How is a hash table used?

Zend engine defines a number of API functions for use in hash tables. A low-level hash table function preview can be found in the zend_hash.h file. In addition, Zend engine zend_API.h defines a slightly more advanced API in the file.

We don't have enough time to talk about all the functions, but at least we can look at some instance functions to see how it works. We will use it array_fill_keys as an instance function.

Using the techniques mentioned in the second section, you can easily find the functions defined in the ext/standard/array.c file. Now, let's take a quick look at this function.

Like most functions, the top of a function has a bunch of variables defined, and then the function is called zend_parse_parameters :

Zval *keys, *val, * *entry; Hashposition Pos; if " AZ ", &keys, &val) = = FAILURE)    {return;}

Obviously, the az parameter indicates that the first parameter type is an array (that is, a variable keys ), and the second argument is any zval (that is, a variable val ).

After parsing the parameters, the returned array is initialized:

/**/array_init_size (Return_value, Zend_hash_num_elements (z_arrval_p (keys)));

This line contains three important parts of the array API:

1, z_arrval_p macro extracts the value from the Zval to the hash table.

2. Zend_hash_num_elements extract the number of hash table elements (nnumofelements attribute).

3. Array_init_size Initializes an array with a size variable.

Therefore, this line initializes the array into the variable using the same size as the key-value array return_value .

The size here is just an optimization scenario. Functions can also be called only array_init(return_value) , so that as more and more elements are added to the array, PHP resets the size of the array multiple times. By specifying a specific size, PHP allocates the correct memory space at the outset.

After the array is initialized and returned, the function uses the same code structure as the following, using the while loop variable keys array:

ZEND_HASH_INTERNAL_POINTER_RESET_EX (Z_arrval_p (keys), &POS);  while (ZEND_HASH_GET_CURRENT_DATA_EX (Z_arrval_p (keys), (void * *) &entry, &pos) = = SUCCESS    ) { // some code &POS);}     

This can be easily translated into PHP code:

Reset ($keys);  while (null$entrycurrent ($keys)) {    //  some code    Next($keys);}

As in the following:

foreach ($keysas$entry) {    //  some code}

The only difference is that the traversal of C does not use an internal array pointer, but uses its own POS variable to store the current position.

The code inside the loop is divided into two branches: one for numeric keys and another for other keys. The branch of a numeric key value has only the following two lines of code:

zval_add_ref(&val);zend_hash_index_update(Z_ARRVAL_P(return_value), Z_LVAL_PP(entry), &val, sizeof(zval *), NULL);

This looks too straightforward: the reference to the first value is incremented (adding a value to the Hashtable means adding another reference to it), and then the value is inserted into the hash table. zend_hash_index_updatethe macro parameters are the hash table that needs to be updated, the Z_ARRVAL_P(return_value) integer subscript Z_LVAL_PP(entry) , the value &val , the size of the value, sizeof(zval *) and the target pointer (which we are not concerned with, therefore NULL ).

The branch of a non-numeric subscript is slightly more complicated:

 zval key, *key_ptr = *entry;  if     (Z_TYPE_PP (entry)!= is_string) {key  = **entry;    Zval_copy_ctor ( &key);    Convert_to_string ( &key); Key_ptr  = &key;} Zval_add_ref ( &val); Zend_symtable_update (Z_arrval_p (return_value), z_ Strval_p (Key_ptr), Z_strlen_p (key_ptr)  + 1 , &val, sizeof  (zval * if  (Key_ptr! = *entry) {zval_dtor ( &key);}  

First, use to convert_to_string convert the key value to a string (unless it is already a string). Before that, it entry is copied to the new key variable. key = **entrythis line is implemented. In addition, zval_copy_ctor functions are called, otherwise complex structures (such as strings or arrays) are not copied correctly.

The above copy operation is necessary because the type conversion is guaranteed to not change the original array. Without a copy operation, the cast not only modifies the local variables, but also modifies the values in the array of key values (obviously, this is very surprising to the user).

Obviously, after the loop is over, the copy operation needs to be removed again, and that's the zval_dtor(&key) job. zval_ptr_dtor zval_dtor The difference is that the zval_ptr_dtor zval variable is destroyed only when the refcount variable is 0 o'clock, and zval_dtor it is destroyed immediately, instead refcount of relying on the value. This is why you see the use of zval_pte_dtor temporary variables using "normal" variables, which are zval_dtor not used elsewhere. Furthermore, the zval_ptr_dtor content of Zval will be released after destruction without zval_dtor . Because we do not have malloc() anything, so we do not need free() , so in this regard, zval_dtor made the right choice.

Now look at the remaining two lines (the important two lines ^ ^):

Zval_add_ref (&1sizeof(Zval *), NULL);

This is very similar to the operation after the numeric key branch is completed. The difference is that the call is now zend_symtable_update instead of the zend_hash_index_update key value string and its length is passed.

Symbol table

The function of "normal" insert string key value into a hash table is zend_hash_update , but here it is used zend_symtable_update . What difference do they have?

The symbol table is simply a special type of hash table, which is used in arrays. It differs from the original hash table in how he handles the numeric key values: In the symbol table, "123" and 123 are considered to be the same. So, if you store a value in $array["123"], you can get it later using $array[123].

The bottom layer can be implemented in two ways: either use "123" to save 123 and "123", or use 123来 to save the two key values. Obviously PHP chose the latter (because the integer is faster and takes up less space than the string type).

If you accidentally insert data after using "123" instead of casting to 123, you will find some interesting things in the symbol table. A cast using an array-to-object is as follows:

New stdClass; $obj->{123"foo"= (array) $obj; Var_dump ($ arr[123//  Undefined offset:123var_dump ($arr ["123"  //  Undefined offset:123

Object properties are always saved using string key values, although they are numbers. So $obj->{123} = ‘foo‘ this line of code actually saves the ' foo ' variable to the ' 123 ' subscript. When using an array cast, this value is not changed. However $arr[123] , when and $arr["123"] both want to access the value of 123 (not an existing "123" subscript), the error is thrown. So, congratulations, you created a hidden array element.

Next section

The next section will be published again in Ircmaxell's blog. The next article describes how objects and classes work internally.

Understanding the implementation of the array within PHP (PHP source for PHP Developers-part fourth)