"Numpy" Memory analysis _numpy structured arrays

Three, NumPy structure array

"Numpy" Memory Analysis _numpy.dtype Memory Data parsing method guide

Np.dtype can be used to construct an array of structures, Numpy.ndarray.base will return the memory owner's information, the document is as follows,

Help on Getset descriptor Numpy.ndarray.base:

Base
Base Object If memoryis from some and other object.

Examples
--------
The base of an array that owns it memory is None:

>>> x = Np.array ([1,2,3,4])
>>> X.base is None
True

Slicing creates a view, whose memory is shared with x:

>>> y = x[2:]
>>> Y.base is X
True

1. Building an array of structures

PersonType = Np.dtype ({    ' names ': [' name ', ' age ', ' weight ', ' height '],    ' formats ': [' S30 ', ' I ', ' f ', ' F ']}, align= True) A = Np.array ([' Zhang ', 32,72.5,167),              (' Wang ', 24,65,170)],dtype=persontype) a[' age '].base

Array ([B ' Zhang ', 32, 72.5, 167.),

(b ' Wang ', 24, 65., 170.)],

dtype={' names ': [' name ', ' age ', ' weight ', ' height '],

' Formats ': [' S30 ', ' <i4 ', ' <f4 ', ' <f4 '],

' Offsets ': [0,32,36,40],

' ItemSize ': 44,

' aligned ': True})

2, the difference between the advanced slice and the ordinary slice

In []: A.basein [+]: A[0].basein []: a[:1].baseout[28]: Array ([123,   4,   5,   6, approx  ]) in []: a[[0,1]] . Basein [[+]: A.base is noneout[30]: Truein [[+]: A[0].base is noneout[31]: Truein [+]: A[:1].base is noneout[32]: Falsei n [[]: A[[0,1]].base is noneout[33]: True

By the visible advanced slice will open up new memory , copy the cut out of the data, this is because this irregular memory access using the original memory structure is very inefficient (logical adjacent element memory is not adjacent, standard access due to fixed the start and step equivalent to access adjacent elements, so more efficient), The copy is a contiguous array of memory.

3. Advanced slicing method that does not open new memory

Back to the structure array on the top bar,

Print (a[' age '].base was a) print (a[[' age ', ' height ']].base is None)

True

True

We do this by specifying the memory resolution method , which does not open up new memory, resolves the original memory to an array of structures specified by the Advanced slice,

def fields_view (arr, fields): Dtype2 = Np.dtype ({name:arr.dtype.fields[name] For name in fields}) # print (dtype2) # {' Names ': [' age ', ' weight '], ' formats ': [' <i4 ', ' <f4 '], ' offsets ': [32,36], ' ItemSize ': + # print ([(Name,arr.dtype.fields[name]) for name in Fields]) # [(' Age ', (Dtype (' int32 '), +), (' Weigh T ', (Dtype (' float32 '), +)] # print (arr.strides) # (,) return Np.ndarray (Arr.shape, dtype2, arr, 0, Arr.stride          s) ' Ndarray (Shape, dtype=float, Buffer=none, offset=0, | Strides=none, Order=none) Parameter type Action shape int tuple multidimensional array Shapes Dtype data-type array of elements of type buffer used to initialize the array in the Bufferoffset int buffer The offset of the first data of the initialized array strides int tuple increments by 1 o'clock for each axis, the number of bytes added by the data pointer in memory order ' C ' or ' F ' C ': line precedence; ' F ': Column precedence ' ' V = Fields_view (A, [' Age ', ' Weight ']) print (V.base is a) v[' age ' + = 10print (' + + ' *10) print (v) print (v.dtype) print (' + + ' V.dtype.fields) Print (a) print (A.dtype) print (a.dtype.fields) 

true++++++++++++++++++++++++++++++[(72.5,  ' itemsize ':+} {' Age ': (Dtype (' int32 '), 32), ' Weight ': (Dtype (' float32 '), 72.5)}++++++++++++++++++++++++++++++[(b ' Zhang ',  167.) (b ' Wang ', '  itemsize ':aligned, ' true}{': ' Name ': (Dtype (' S30 '), 0), ' Age ': (Dtype (' int32 ') , +), ' weight ': (Dtype (' float32 '), ' height ': (Dtype (' float32 '), 40)}

Notice here. Dtype's ' itemsize ' parameter, which means adding a (row) of data, how many bytes of memory has been added, because the ' offsets ' offset information is saved, the dtype we generate shows a sparse structure , But each row does not have an extra tail, because the empty element is generated by the gap in the real element's record offset.

4. View

"Numpy" numpy.ndarray.view_ array view _reshape, array slicing, array memory opening analysis

If the itemsize of the two are the same, we can use Array.view (dtype=dtype) directly to specify the display view of the array memory data, and the above code modifies it.

def fields_view (arr, fields):    dtype2 = Np.dtype ({name:arr.dtype.fields[name] for name in fields})    return Np.ndarray (Arr.shape, dtype2, arr, 0, arr.strides) v = Fields_view (A, [' age ', ' height ']) a.view (Dtype=v.dtype)

Array ([  167.)  ],       dtype={' names ': [' age ', ' height '], ' formats ': [' <i4 ', ' <f4 '], ' Offsets ': [32,40], ' itemsize ': 44})

When we last offset to the end of the line is the property ' height ', the two itemsize equal, you can specify the view method resolution,

Print (V.dtype) # {' Names ': [' age ', ' height '], ' formats ': [' <i4 ', ' <f4 '], ' offsets ': [32,40], ' itemsize ': 44}print ( A.dtype) # {' Names ': [' name ', ' age ', ' weight ', ' height '], ' formats ': [' S30 ', ' <i4 ', ' <f4 ', ' <f4 '], ' offsets ': [ 0,32,36,40], ' itemsize ':, ' aligned ': True}

This is because the view is used to parse the memory data parsing, for the structure array is the resolution of each row, this parsing method (that is, the input parameters of our own dtype) must certainly and memory data description parameters (that is, memory dependent array of dtype) consistent, otherwise the parsing data can not be aligned.

A.view (Dtype=v.dtype)

Array ([  167.)  ],       dtype={' names ': [' age ', ' height '], ' formats ': [' <i4 ', ' <f4 '], ' Offsets ': [32,40], ' itemsize ': 44})

"Numpy" Memory analysis _numpy structured arrays