python進程池multiprocessing.Pool和線程池multiprocessing.dummy.Pool實例

進程池：

進程池的使用有四種方式：apply_async、apply、map_async、map。其中apply_async和map_async是異步的，也就是啓動進程函數以後會繼續執行後續的代碼不用等待進程函數返回。apply_async和map_async方式提供了一寫獲取進程函數狀態的函數： ready()、 successful()、 get()。

PS： join()語句要放在 close()語句後面。

 

實例代碼以下：

# -*- coding: utf-8 -*-

import multiprocessing
import time


def func(msg):
    print('msg: ', msg)
    time.sleep(1)
    print('********')
    return 'func_return: %s' % msg

if __name__ == '__main__':
    # apply_async
    print('\n--------apply_async------------')
    pool = multiprocessing.Pool(processes=4)
    results = []
    for i in range(10):
        msg = 'hello world %d' % i
        result = pool.apply_async(func, (msg, ))
        results.append(result)
    print('apply_async: 不堵塞')

    for i in results:
        i.wait()  # 等待進程函數執行完畢

    for i in results:
        if i.ready():  # 進程函數是否已經啓動了
            if i.successful():  # 進程函數是否執行成功
                print(i.get())  # 進程函數返回值

    # apply
    print('\n--------apply------------')
    pool = multiprocessing.Pool(processes=4)
    results = []
    for i in range(10):
        msg = 'hello world %d' % i
        result = pool.apply(func, (msg,))
        results.append(result)
    print('apply: 堵塞')  # 執行完func才執行該句
    pool.close()
    pool.join()  # join語句要放在close以後
    print(results)

    # map
    print('\n--------map------------')
    args = [1, 2, 4, 5, 7, 8]
    pool = multiprocessing.Pool(processes=5)
    return_data = pool.map(func, args)
    print('堵塞')  # 執行完func才執行該句
    pool.close()
    pool.join()  # join語句要放在close以後
    print(return_data)

    # map_async
    print('\n--------map_async------------')
    pool = multiprocessing.Pool(processes=5)
    result = pool.map_async(func, args)
    print('ready: ', result.ready())
    print('不堵塞')
    result.wait()  # 等待全部進程函數執行完畢

    if result.ready():  # 進程函數是否已經啓動了
        if result.successful():  # 進程函數是否執行成功
            print(result.get())  # 進程函數返回值

　　

線程池：

線程池的使用方式和進程池相似。

 

實例代碼以下：

# -*- coding: utf-8 -*-

from multiprocessing.dummy import Pool as ThreadPool
import time


def fun(msg):
    print('msg: ', msg)
    time.sleep(1)
    print('********')
    return 'fun_return %s' % msg


# map_async
print('\n------map_async-------')
arg = [1, 2, 10, 11, 18]
async_pool = ThreadPool(processes=4)
result = async_pool.map_async(fun, arg)
print(result.ready())  # 線程函數是否已經啓動了
print('map_async: 不堵塞')
result.wait()  # 等待全部線程函數執行完畢
print('after wait')
if result.ready():  # 線程函數是否已經啓動了
    if result.successful():  # 線程函數是否執行成功
        print(result.get())  # 線程函數返回值

# map
print('\n------map-------')
arg = [3, 5, 11, 19, 12]
pool = ThreadPool(processes=3)
return_list = pool.map(fun, arg)
print('map: 堵塞')
pool.close()
pool.join()
print(return_list)

# apply_async
print('\n------apply_async-------')
async_pool = ThreadPool(processes=4)
results =[]
for i in range(5):
    msg = 'msg: %d' % i
    result = async_pool.apply_async(fun, (msg, ))
    results.append(result)

print('apply_async: 不堵塞')
# async_pool.close()
# async_pool.join()
for i in results:
    i.wait()  # 等待線程函數執行完畢

for i in results:
    if i.ready():  # 線程函數是否已經啓動了
        if i.successful():  # 線程函數是否執行成功
            print(i.get())  # 線程函數返回值

# apply
print('\n------apply-------')
pool = ThreadPool(processes=4)
results =[]
for i in range(5):
    msg = 'msg: %d' % i
    result = pool.apply(fun, (msg, ))
    results.append(result)

print('apply: 堵塞')
print(results)

　　

計算多的用多進程

io多的用多線程