tornado解決高併發的初步認識牽扯出的一些問題

時間 2019-11-19

原文原文鏈接

#!/bin/env python
# -*- coding:utf-8 -*-
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web

import tornado.gen
from tornado.concurrent import run_on_executor
from concurrent.futures import ThreadPoolExecutor
import time
from tornado.options import define, options
define("port", default=8000, help="run on the given port", type=int)


class SleepHandler(tornado.web.RequestHandler):
    executor = ThreadPoolExecutor(2)

    @tornado.web.asynchronous
    @tornado.gen.coroutine
    def get(self):
        # 假如你執行的異步會返回值被繼續調用能夠這樣(只是爲了演示),不然直接yield就行
        res = yield self.sleep()
        self.write("when i sleep %s s" % res)
        self.finish()

    @run_on_executor
    def sleep(self):
        time.sleep(5)
        return 5


class JustNowHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("i hope just now see you")


if __name__ == "__main__":
    tornado.options.parse_command_line()
    app = tornado.web.Application(handlers=[
            (r"/sleep", SleepHandler), (r"/justnow", JustNowHandler)])
    http_server = tornado.httpserver.HTTPServer(app)
    http_server.listen(options.port)
    tornado.ioloop.IOLoop.instance().start()

上面的代碼示例轉載自：https://blog.csdn.net/qq_28893679/article/details/69437496python

原本經過線程池我以爲就能夠實現高併發了，可是這位道友的@tornado.gen.coroutine這個裝飾器，讓我從新認識了協程，以前我一直覺得協程是用來在單線程中解決io多路服用的，其實我還忽略了一個重要的地方，就是協程會主動讓出CPU資源，這點我竟然會遺忘，哎，天殺的，因此協程還有個重要的做用就是主動讓出CPU資源。web

@tornado.web.asynchronous的源碼和本身瞎胡分析數據結構

@asynchronous源碼:
def asynchronous(method):
    """Wrap request handler methods with this if they are asynchronous.

    This decorator is for callback-style asynchronous methods; for
    coroutines, use the ``@gen.coroutine`` decorator without
    ``@asynchronous``. (It is legal for legacy reasons to use the two
    decorators together provided ``@asynchronous`` is first, but
    ``@asynchronous`` will be ignored in this case)

    This decorator should only be applied to the :ref:`HTTP verb
    methods <verbs>`; its behavior is undefined for any other method.
    This decorator does not *make* a method asynchronous; it tells
    the framework that the method *is* asynchronous.  For this decorator
    to be useful the method must (at least sometimes) do something
    asynchronous.

    If this decorator is given, the response is not finished when the
    method returns. It is up to the request handler to call
    `self.finish() <RequestHandler.finish>` to finish the HTTP
    request. Without this decorator, the request is automatically
    finished when the ``get()`` or ``post()`` method returns. Example:

    .. testcode::

       class MyRequestHandler(RequestHandler):
           @asynchronous
           def get(self):
              http = httpclient.AsyncHTTPClient()
              http.fetch("http://friendfeed.com/", self._on_download)

           def _on_download(self, response):
              self.write("Downloaded!")
              self.finish()

    .. testoutput::
       :hide:

    .. versionchanged:: 3.1
       The ability to use ``@gen.coroutine`` without ``@asynchronous``.

    .. versionchanged:: 4.3 Returning anything but ``None`` or a
       yieldable object from a method decorated with ``@asynchronous``
       is an error. Such return values were previously ignored silently.
    """
    # Delay the IOLoop import because it's not available on app engine.
    from tornado.ioloop import IOLoop

    @functools.wraps(method)
    def wrapper(self, *args, **kwargs):
        self._auto_finish = False
        with stack_context.ExceptionStackContext(
                self._stack_context_handle_exception):
            result = method(self, *args, **kwargs)
            if result is not None:
                result = gen.convert_yielded(result)

                # If @asynchronous is used with @gen.coroutine, (but
                # not @gen.engine), we can automatically finish the
                # request when the future resolves.  Additionally,
                # the Future will swallow any exceptions so we need
                # to throw them back out to the stack context to finish
                # the request.
                def future_complete(f):
                    f.result()
                    if not self._finished:
                        self.finish()
                IOLoop.current().add_future(result, future_complete)
                # Once we have done this, hide the Future from our
                # caller (i.e. RequestHandler._when_complete), which
                # would otherwise set up its own callback and
                # exception handler (resulting in exceptions being
                # logged twice).
                return None
            return result
    return wrapper
	
蹩腳的翻譯:
def asynchronous(method):
	若是處理請求的方法是異步的，就用此方法來包裝它(它代指get,post等方法)
	這個裝飾器用於回調類型的異步方法;對於協程,請用"@gen.coroutine"裝飾器而不是"@asynchronous".(若是要同時合法的使用這兩個裝飾器,前提是"@asynchronous"要放在第一個也就是最外面,但在這種狀況下"@asynchronous"將被忽略)
	這個裝飾器應該只能被應用到:ref:'HTTP 動態方法(get...)';它的行爲對任何其它的方法都是未定義的.這個裝飾器不製做異步方法;它告訴框架這個方法是異步的.對於這個裝飾器包裝的方法必定(至少一段時間)要作一些異步的事情.
	若是給出了這個裝飾器,被包裝的方法已經return的時候響應尚未完成.完不完成取決於處理請求的方法去調用"self.finish()<RequestHandler.finish>"去結束此次HTTP請求.沒有這個裝飾器,請求會在"get()"或"post()"方法return的時候結束.例如：

	.. 測試代碼::

	   class MyRequestHandler(RequestHandler):
		   @asynchronous
		   def get(self):
			  http = httpclient.AsyncHTTPClient()
			  http.fetch("http://friendfeed.com/", self._on_download)

		   def _on_download(self, response):
			  self.write("Downloaded!")
			  self.finish()

	.. 測試輸出::
		:隱藏:
		
	.. 版本改變:3.1
		能用"@gen.coroutine"實現就不用"@asynchronous"

	.. 版本改變:4.3 在被該裝飾器包裝的方法中返回"None"或者一個yieldable對象是一個錯誤.這種返回值在以前是默認忽略的.
	# 延遲IOLoop導入由於它不能在app引擎上用
	from tornado.ioloop import IOLoop

	@functools.wraps(method)
	def wrapper(self,*args,**kwargs):
		self._auto_finish = False
		with stack_context.ExceptionStackContext(
				self._stack_context_handle_exception):
			result = method(self,*args,**kwargs)
			if result is not None:
				result = gen.convert_yielded(result)
				
				#若是 @asynchronous 是和@gen.coroutine(但不是@gen.engine)一塊兒用的,咱們
				#能自動完成請求當future對象resolves時(我理解處於解決狀態).此外,future對象
				#將吞下全部異常,所以咱們須要讓他們退出堆棧上下文去完成請求.
				def future_complete(f):
					f.result()
					if not self._finished:
						self.finish()
				IOLoop.current().add_future(result,future_complete)
				#一旦咱們完成了這一步，隱藏來自調用者的Future
				#(即RequestHandler._when_complete),
				#不然它會設置本身的回調和異常處理程序(致使異常被記錄兩次).
				return None
			return result
	return wrapper

隨後我又開始看了@gen.conroutine而後在源碼裏發現了一個神奇的模塊types模塊，大開這個模塊後發現了各類類型的定義，而後我又莫名奇妙的想到了數據結構和數據類型，接下來就一發不可收拾了，簡明python教程中說，數據結構是一種結構，可以將一些數據聚合在一塊兒，也就是用來存儲一系列相關數據的集合。python內置的數據結構有四種列表，集合，字典，元組。而後我又想起python的數據類型整形，字符串，列表，字典，集合，元組。到這裏我開始犯糊塗了，怎麼回事，數據結構也能夠是數據類型麼，活着數據類型也能夠同時是數據結構麼。查閱了不少資料(就是本身想了半天)後我總結除了本身的理解：併發

　　數據結構=數據+組合方式，就是把好多數據多放在一塊兒了，對吧
　　數據類型=數據+處理方式，就是把原生數據(0101)加工處理了一下，變成了現有的數據app

不要打我。。。框架