Python進階:生成器--懶人版本的迭代器

從容器、可迭代對象談起

　　全部的容器都是可迭代的（iterable）,迭代器提供了一個next方法。iter()返回一個迭代器，經過next()函數能夠實現遍歷。

def is_iterable(param):
    try: 
        iter(param) 
        return True
    except TypeError:
        return False

params = [
    1234,
    '1234',
    [1, 2, 3, 4],
    set([1, 2, 3, 4]),
    {1:1, 2:2, 3:3, 4:4},
    (1, 2, 3, 4)
]
    
for param in params:
    print('{} is iterable? {}'.format(param, is_iterable(param)))

########## 輸出 ##########

# 1234 is iterable? False
# 1234 is iterable? True
# [1, 2, 3, 4] is iterable? True
# {1, 2, 3, 4} is iterable? True
# {1: 1, 2: 2, 3: 3, 4: 4} is iterable? True
# (1, 2, 3, 4) is iterable? True

　　除了數字外,其餘數據結構都是可迭代的。

生成器是什麼

　　生成器是懶人版本的迭代器。例:

import os
import psutil

#顯示當前 python 程序佔用的內存大小
def show_memory_info(hint):
    pid = os.getpid()
    p = psutil.Process(pid)
    
    info = p.memory_full_info()
    memory = info.uss / 1024. / 1024
    print('{} memory used: {} MB'.format(hint, memory))

def test_iterator():
    show_memory_info('initing iterator')
    list_1 = [i for i in range(100000000)]
    show_memory_info('after iterator initiated')
    print(sum(list_1))
    show_memory_info('after sum called')

def test_generator():
    show_memory_info('initing generator')
    list_2 = (i for i in range(100000000))
    show_memory_info('after generator initiated')
    print(sum(list_2))
    show_memory_info('after sum called')

test_iterator()
test_generator()
%time test_iterator()
%time test_generator()

######### 輸出 ##########

initing iterator memory used: 48.9765625 MB
after iterator initiated memory used: 3920.30078125 MB
4999999950000000
after sum called memory used: 3920.3046875 MB
Wall time: 17 s
initing generator memory used: 50.359375 MB
after generator initiated memory used: 50.359375 MB
4999999950000000
after sum called memory used: 50.109375 MB
Wall time: 12.5 s

　　[i for i in range(100000000)] 聲明瞭一個迭代器，每一個元素在生成後都會保存到內存中，佔用了巨量的內存。(i for i in range(100000000)) 初始化了一個生成器，能夠看到，生成器並不會像迭代器同樣佔用大量的內存，相比於 test_iterator()，test_generator()函數節省了一次生成一億個元素的過程。在調用next()的時候，纔會生成下一個變量.

生成器能玩啥花樣

　　數學中有一個恆等式，(1 + 2 + 3 + ... + n)^2 = 1^3 + 2^3 + 3^3 + ... + n^3，用如下代碼表達

def generator(k):
    i = 1
    while True:
        yield i ** k
        i += 1

gen_1 = generator(1)
gen_3 = generator(3)
print(gen_1)
print(gen_3)

def get_sum(n):
    sum_1, sum_3 = 0, 0
    for i in range(n):
        next_1 = next(gen_1)
        next_3 = next(gen_3)
        print('next_1 = {}, next_3 = {}'.format(next_1, next_3))
        sum_1 += next_1
        sum_3 += next_3
    print(sum_1 * sum_1, sum_3)

get_sum(8)

########## 輸出 ##########

# <generator object generator at 0x000001E70651C4F8>
# <generator object generator at 0x000001E70651C390>
# next_1 = 1, next_3 = 1
# next_1 = 2, next_3 = 8
# next_1 = 3, next_3 = 27
# next_1 = 4, next_3 = 64
# next_1 = 5, next_3 = 125
# next_1 = 6, next_3 = 216
# next_1 = 7, next_3 = 343
# next_1 = 8, next_3 = 512
# 1296 1296

　　generator()這個函數，它返回了一個生成器，當運行到yield i ** k時，暫停並把i ** k做爲next()的返回值。每次調用next(gen)時，暫停的程序會啓動並往下執行，並且i的值也會被記住，繼續累加，最後next_1爲8，next_3爲512.

　　仔細查看這個示例，發現迭代器是一個有限集合，生成器則能夠成爲一個無限集。調用next()，生成器根據運算會自動生成新的元素，而後返回給你，很是便捷。

　　再來看一個問題：給定一個list和一個指定數字，求這個數字在list中的位置:

#常規寫法
def index_normal(L, target):
    result = []
    for i, num in enumerate(L):
        if num == target:
            result.append(i)
    return result

print(index_normal([1, 6, 2, 4, 5, 2, 8, 6, 3, 2], 2))

########## 輸出 ##########

[2, 5, 9]


#生成器寫法
def index_generator(L, target):
    for i, num in enumerate(L):
        if num == target:
            yield i

print(list(index_generator([1, 6, 2, 4, 5, 2, 8, 6, 3, 2], 2)))

######### 輸出 ##########

[2, 5, 9]

　　再看一例子:

　　查找子序列：給定兩個字符串a,b,查找字符串a是否字符串b的子序列,所謂子序列,即一個序列包含在另外一個序列中而且順序一致。[1,3,5]是[1,2,3,4,5]子序列，而[1,4,3]不是 [1,2,3,4,5] 的子序列.

　　算法:分別用兩個指針指向兩個字符串的頭，而後日後移動找出相同的值，若是其中一個指針走完了整個字符串也沒有相同的值，則不是子序列

def is_subsequence(a, b):
    b = iter(b)
    return all(i in b for i in a)

print(is_subsequence([1, 3, 5], [1, 2, 3, 4, 5]))
print(is_subsequence([1, 4, 3], [1, 2, 3, 4, 5]))

######### 輸出 ##########

True
False

　　下面代碼爲上面代碼的演化版本

def is_subsequence(a, b):
    b = iter(b)
    print(b)

    gen = (i for i in a)
    print(gen)

    for i in gen:
        print(i)

    gen = ((i in b) for i in a)
    print(gen)

    for i in gen:
        print(i)

    return all(((i in b) for i in a))

print(is_subsequence([1, 3, 5], [1, 2, 3, 4, 5]))
print(is_subsequence([1, 4, 3], [1, 2, 3, 4, 5]))

########## 輸出 ##########

# <list_iterator object at 0x000001E7063D0E80>
# <generator object is_subsequence.<locals>.<genexpr> at 0x000001E70651C570>
# 1
# 3
# 5
# <generator object is_subsequence.<locals>.<genexpr> at 0x000001E70651C5E8>
# True
# True
# True
# False
# <list_iterator object at 0x000001E7063D0D30>
# <generator object is_subsequence.<locals>.<genexpr> at 0x000001E70651C5E8>
# 1
# 4
# 3
# <generator object is_subsequence.<locals>.<genexpr> at 0x000001E70651C570>
# True
# True
# False
# False

　　首先iter(b)把b轉爲迭代器。目的是內部實現next函數，(i for i in a) 會產生一個生成器 ，一樣((i in b) for i in a)也是。而後(i in b)等階於:

while True:
    val = next(b)
    if val == i:
        yield True

　　這裏很是巧妙地利用生成器的特性，next()函數運行的時候，保存了當前的指針。好比下面這個示例

b = (i for i in range(5))

print(2 in b)
print(4 in b)
print(3 in b)

########## 輸出 ##########

True
True
False

 參考

　　極客時間《Python核心技術與實戰》專欄