Queue

queue隊列

queue is especially useful in threaded programming when information must be exchanged safely between multiple threads.

class queue.Queue(maxsize=0) #先入先出 

class queue.LifoQueue(maxsize=0) #last in fisrt out 

class queue.PriorityQueue(maxsize=0) #存儲數據時可設置優先級的隊列

Constructor for a priority queue. maxsize is an integer that sets the upperbound limit on the number of items that can be placed in the queue. Insertion will block once this size has been reached, until queue items are consumed. If maxsize is less than or equal to zero, the queue size is infinite.

The lowest valued entries are retrieved first (the lowest valued entry is the one returned by sorted(list(entries))[0]). A typical pattern for entries is a tuple in the form: (priority_number, data).

exception queue.Empty

Exception raised when non-blocking get() (or get_nowait()) is called on a Queue object which is empty.

exception queue.Full

Exception raised when non-blocking put() (or put_nowait()) is called on a Queue object which is full.

Queue. qsize ()

Queue. empty () #return True if empty  

Queue. full () # return True if full 

Queue. put (item, block=True, timeout=None)

Put item into the queue. If optional args block is true and timeout is None (the default), block if necessary until a free slot is available. If timeout is a positive number, it blocks at most timeout seconds and raises the Full exception if no free slot was available within that time. Otherwise (block is false), put an item on the queue if a free slot is immediately available, else raise the Full exception (timeout is ignored in that case).

Queue. put_nowait (item)

Equivalent to put(item, False).

Queue. get (block=True, timeout=None)

Remove and return an item from the queue. If optional args block is true and timeout is None (the default), block if necessary until an item is available. If timeout is a positive number, it blocks at most timeout seconds and raises the Empty exception if no item was available within that time. Otherwise (block is false), return an item if one is immediately available, else raise the Empty exception (timeout is ignored in that case).

Queue. get_nowait ()

Equivalent to get(False).

Two methods are offered to support tracking whether enqueued tasks have been fully processed by daemon consumer threads.

Queue. task_done ()

Indicate that a formerly enqueued task is complete. Used by queue consumer threads. For each get() used to fetch a task, a subsequent call to task_done() tells the queue that the processing on the task is complete.

If a join() is currently blocking, it will resume when all items have been processed (meaning that a task_done() call was received for every item that had been put() into the queue).

Raises a ValueError if called more times than there were items placed in the queue.

Queue. join () block直到queue被消費完畢

生產者消費者模型

在併發編程中使用生產者和消費者模式可以解決絕大多數併發問題。該模式經過平衡生產線程和消費線程的工做能力來提升程序的總體處理數據的速度。

爲何要使用生產者和消費者模式

在線程世界裏，生產者就是生產數據的線程，消費者就是消費數據的線程。在多線程開發當中，若是生產者處理速度很快，而消費者處理速度很慢，那麼生產者就必須等待消費者處理完，才能繼續生產數據。一樣的道理，若是消費者的處理能力大於生產者，那麼消費者就必須等待生產者。爲了解決這個問題因而引入了生產者和消費者模式。

什麼是生產者消費者模式

生產者消費者模式是經過一個容器來解決生產者和消費者的強耦合問題。生產者和消費者彼此之間不直接通信，而經過阻塞隊列來進行通信，因此生產者生產完數據以後不用等待消費者處理，直接扔給阻塞隊列，消費者不找生產者要數據，而是直接從阻塞隊列裏取，阻塞隊列就至關於一個緩衝區，平衡了生產者和消費者的處理能力。

隊列的一些應用

>>>import queue
>>>q = queue.Queue()   #能夠加maxsize參數，如加入3，表示最多隻能添加3個
>>>q.put("q1")              #有block=True和timeout=None兩個參數，若是設置了block，則當隊列滿，會拋出一串，get()函數同put同樣
>>>q.put("q2")
>>>q.put("q3")
>>>q.get()
>>>'q1'
>>>q.get()
>>>'q2'
>>>q.get()
>>>'q3'
>>>q.get()    #此時隊列中已經沒有數據了，此時咱們再get，就阻塞了


import queue
>>>q = queue.Queue(3)
>>>q.put("q3",timeout = 3)
>>>q.put("q3",timeout = 3)
>>>q.put("q3",timeout = 3)
>>>q.put("q3",timeout = 3)    #等待3秒，列表仍是滿的話就拋出異常
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\python35\lib\queue.py", line 141, in put
    raise Full

import queue
>>>q = queue.Queue(3)
>>>q.put("q3",block = False)
>>>q.put("q3",block = False)
>>>q.put("q3",block = False)
>>>q.put("q3",block = False)       #若是列表滿，就直接拋出異常，不阻塞
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\python35\lib\queue.py", line 130, in put
    raise Full
queue.Full

　　

兩個生產者消費者的例子

# import queue
#
# q = queue.Queue()
#
# def producer():
#     for i in range(10):
#         q.put("骨頭 %s"%i)
#
#     print("開始等待全部的骨頭被取走...")
#     q.join()                        #阻塞調用線程，直到隊列中全部任務都被處理掉
#     print("全部的骨頭被取完了...")
#
# def consumer(n):
#
#     while q.qsize():
#         print("%s 取到"%n,q.get())
#         q.task_done()    #意味着以前入隊的一個任務已經完成。由隊列的消費者線程調用。每個get()調用獲得一個任務，接下來的task_done()調用告訴隊列該任務已經處理完畢。
#
# t =threading.Thread(target=producer)
# c = threading.Thread(target=consumer,args=("lll",))
# t.start()
# c.start()


import time,random
import queue,threading
q = queue.Queue()
def Producer(name):
  count = 0
  while count <20:
    time.sleep(random.randrange(3))
    q.put(count)
    print('Producer %s has produced %s baozi..' %(name, count))
    count +=1
def Consumer(name):
  count = 0
  while count <20:
    time.sleep(random.randrange(4))
    if not q.empty():
        data = q.get()
        print(data)
        print('\033[32;1mConsumer %s has eat %s baozi...\033[0m' %(name, data))
    else:
        print("-----no baozi anymore----")
    count +=1
p1 = threading.Thread(target=Producer, args=('A',))
c1 = threading.Thread(target=Consumer, args=('B',))
p1.start()
c1.start()