iterators, iteration objects, generators

can iterate over objects Iterative Protocol

We are now analyzing the reason for the result, which can be "iterative" for the loop, but if you're thinking about, for how do you know who is iterative?

If we write a data type ourselves and hope that something in this data type can also be taken out of a for by one, then we must meet the requirements for. This requirement is called " Agreement ".

a requirement that can be met by an iteration is called an iterative protocol. * the definition of an iterative protocol is very simple, that is, the internal implementation of the Iter method. *

If this object has the _ iter _ () method, this object is an iterative object

If ' __iter__ ' in Dir (str)

easy to understand : an object that can be iterated by a for loop is an iterative object.

Isinstance () Determines whether an object is a Iterable object:

>>> from collections import iterable
>>> isinstance ([], iterable)
True
>>> Isinstance ({}, iterable)
true
>>> isinstance (' abc ', iterable)
true
>>> Isinstance ((x for X in range), iterable)
True
>>> isinstance (iterable)
False

iterators What is an iterator?

An iterator is nothing more than a container object that implements an iterator protocol. It is based on two methods: Next returns the container's next item _ iter _ returns the iterator itself

i = iter (' ASD ')

next (i)
>>> ' A '

next (i)
>>> ' s '

next (i)
>>> ' d '

Next (i)

Traceback (most recent):

  File "<stdin>", line 1, in <module>

stopiteration

When the sequence is facilitated, a Stopiteration exception is thrown. This will make iterators compatible with loops because they will catch this exception to stop the loop. To create a custom iterator, you can write a class with the next method Why use the iterator "streaming" Data processing method * Less memory consumption

For example, when working with a file, all the data is taken out of the memory for processing, which can cause the program to consume a lot of memory

Generally we will part of the content of the file processing:

For Text_line in open ("Xx.txt"):  
    print text_line  

open ("Xx.txt") returns an iterative object, so you can incrementally process the contents of the file, which reads the file by row, and do the processing instead of

directly loading all the files into memory.

Support facilitates processing of data with a for statement

Some of the common types of Python-like arrays, lists, and even strings-are iterative types,

This allows us to facilitate the use of this syntax for the data consumption, do not need to record the index location, human circulation:

For i in [1,2,3,4]:
    print (i)

Advantages of IteratorsVery easy to use, and can only take all the data to save memory space internal of an iterator to iterate over an object:

The object contains the implementation of the __iter () __ method, and the object's ITER function returns an iterator after the call, which contains the implementation iterator for the specific data acquisition :

Contains an implementation of the next method that returns the expected data within the correct range and the error that can throw stopiteration after it has gone out of range to stop the iteration.

The iterator can be used not only for the For loop, but also for the next () function to call and return the following value until the last throw Stopiteration error indicates that the next value cannot be returned.

an object that can be called by the next () function and constantly return the next value is called an iterator: iterator.

generators are iterator objects, but list, dict, str, although iterable, but not iterator

to the list, dict, str iterable into iterator can use the ITER () function:

Iteration Protocol

must have _iter_ methods and _next_ methods. Iterator and Iteration object formal statements:

An object that implements the Iter method is iterative, and an object that implements the next method is an iterator's personal understanding:

Only the ITER method is implemented as an iterative object, and the iterative object can be converted to an iterator by the. _ iter _ () method.

Iterators have Next and _ iter _ methods as long as the two methods exist are iterators. An iterator is a stored container that takes out data whenever the next method is called. and executes only once

that is, the iterator can be used as a data generator Build Device

The essence of the generator is the iterator

Iterators We know are used to iterate over an iterative object, and the generator is used to iterate over the method

We know that there are two kinds of iterators: one is to invoke the method to return directly, one is the iterative object through the implementation of the ITER method, the advantage of the iterator is to save memory.

If in some cases, we also need to save memory, we can only write ourselves. The thing we write ourselves that implements the iterator function is called the generator.

Builder Generator:

Essence: iterators (so the iter method and next method are not required for us to implement)

Features: Lazy operation, developer customization 1. Generator function:

The general function definition, however, returns the result using the yield statement instead of the return statement. The yield statement returns one result at a time, suspending the state of the function in the middle of each result so that the next time it leaves, it continues to execute.

As long as the yield keyword function is the generator function

a function that contains the yield keyword is a generator function. Yield can return a value for us from the function, but yield is different from the execution of Return,return means the end of the program, call the generator function will not get the return of the specific value, but to get an iterative object. Each time you get the value of this iteration object, you can push the execution of the function and get the new return value. Until the function execution ends.

Features:

After the function is called, the function does not execute, and returns a generator
to fetch a value each time the next method is called
until the last one is finished, and the error occurs at execution next.

several ways to take values from the generator

# Next
#
for # cast for # data type: Memory consumption

Example 1:

1 traverse the file to return the file name, and to calculate the size of the file

Import OS
def traverse (dir):
    if Os.path.isdir (dir):
        files = Os.listdir (dir) for
        file in files[::-1]:< C11/>full_name = Os.path.join (dir,file)
            traverse (full_name)
    else:
        yield dir

Call

For I in Traverse (R ' C:\ '):
    print (Os.getsize (i))

Example 2:

Listening for files

Import Time

def tail (filename):
    f = open (filename)
    f.seek (0, 2) #从文件末尾算起 while
    True: line
        = F.readline ()  # Read the new line of text in the file if not lines
        :
            time.sleep (0.1)
            continue
        yield

Call

Tail_g = tail (' tmp ') for line in
tail_g:
    print (line)

Example 3 The adorner of the pre-excitation co-process

Calculate moving Average

Def init (func):
    def Inner (*args,**kwargs):
        g = func (*args,**kwargs)
        next (g) return
        g
    return inner

@init
def averager (): Total
    = 0.0
    count = 0
    average = None while
    1:
        Trem = yield average total =
        term
        count = 1
        average = Total/count

Call

G_avg = Averager () print (g_avg.send) print (g_avg.send) print
(G_avg.send (5))

Understand

We all know that the generator function is executed inside the function only when the next method is called, so that the invocation is easier and easier to understand, and

of course, we can write
G_avg = Averager ( Next (g_avg) print (g_avg.send) print (g_avg.send) print (
g_avg.send (5))

but imagine, If you have n multiple generator functions, we need the N multiple next () method and the statement will look confusing

yield from

Def gen1 (): For
    C in ' AB ':
        yield C to
    I in range (3):
        yield i

print (list (Gen1 ())

def Gen2 ( ):
    yield from ' AB '
    yield from range (3)

print (List (Gen2 ()))

2. Builder expression:

Similar to list derivation, however, the generator returns an object that produces results on demand instead of building a result list list The result of a build is a list syntax:

[x for X in range (Ten)]                  Remove the contents from the array in turn the
equivalent of

x = []
for I in range:
    x.append (i)

[x * 2 for X in range ()]                  remove the contents from the array in turn and Calculates
the equivalent of

x = [] for
i in range:
    x.append (i * 2)

[x for x in range (ten) if x 2 = 0]            from an array of characters  Eligible content takes out the
equivalent of

x = [] for
i in range:
    If I% 2 ==0:
        x.append (i)

[x * 2 for X in range (10) If x 2 = 0] takes the qualifying content out of the        array and calculates the equivalent of

x = [] for
i in range:
    If I% 2 ==0:
        x.append (i * 2)

[x * y for x in range (a) for Y in range ()]   Loop nesting is 
equivalent to 

x = [] for
x in range: for y in
    range:
        x.append (x * y)

[x + 2 = 0 for x in Range (ten) for

x in range:
    if x 2 = 0: return
        True
    else: return
        False

Example One:

30 of all the numbers that can be divisible by 3.

[I for I in range ' If I% 3 is 0]

Case Two:

The square of all the numbers divisible by 3 within 30

[Squared (i) for I (range) if I% 3 is 0]

Example Three:

Find all names containing two ' E ' in the nested list

names = [[' Tom ', ' Billy ', ' Jefferson ', ' Andrew ', ' Wesley ', ' Steven ', ' Joe '],
         [' Alice ', ' Jill ', ' Ana ', ' Wendy ', ' jennif Er ', ' Sherry ', ' Eva '] '

print ([name for LST in names for name in LST if Name.count (' E ') >= 2])

Dictionary Derivation type Example One:

Swap key and value for a dictionary

Mcase = {' A ': ten, ' B ': mcase_frequency}
= {Mcase[k]: K for k in Mcase}
print (mcase_frequency)

Case Two:

Merge case value value, unify K to lowercase

Mcase = {' A ':, ' a ': 7, ' Z ': 3}
mcase_frequency = {k.lower (): Mcase.get (K.lower (), 0) + mcase.get (K.upper () , 0) for K-in Mcase.keys ()}
print (mcase_frequency)

Set Derivation type Example:

Calculates the square of each value in the list, and takes the function with its own weight

Squared = {x**2 for x in [1,-1, 2]}
print (squared)
# output:set ([1, 4])

Of course, you can also pass the method. The use of list derivations is extensive and can be used in conjunction with Python's built-in functions such as Sum max

1. Replace the list resolution [] with a generator expression

2. List resolution and generator expressions are a convenient way of programming, except that the generator expression is more memory-saving

3.Python not only uses the iterator protocol, the for loop becomes more generic. Most built-in functions are also used to access objects using the iterator protocol.