nova的wsgi介紹【WIP】

有關openstack的全部的帖子。

https://www.ustack.com/blog/openstack_hacker/#Nova_Workflow

 網上已經不少的分析文章了：

http://blog.csdn.net/ddl007/article/details/8602731?utm_source=tuicool

http://wenku.baidu.com/view/34892729af45b307e87197ec.html

今天liyong同窗問起pecan的原理。

有必要寫篇博客介紹， nova的wsgi。

 

改博客邊走讀代碼，邊寫，是一個野路子開發者的流程。

 

不習慣使用 tox.

$ cat /usr/bin/pyunittest
python -m unittest `tr / . <<< ${@%.py*}`

1.  首先對wsgi有個形象的瞭解：

   nova/tests/unit/api/test_wsgi.py

 

"""
Test WSGI basics and provide some helper functions for other WSGI tests.
"""

from nova import test

import routes
import webob

from nova import wsgi


class Test(test.NoDBTestCase):

    def test_debug(self):

        class Application(wsgi.Application):
            """Dummy application to test debug."""

            def __call__(self, environ, start_response):
                start_response("200", [("X-Test", "checking")])
                return ['Test result']


        application = wsgi.Debug(Application())
        result = webob.Request.blank('/').get_response(application)
        self.assertEqual(result.body, "Test result")

    def test_router(self):

        class Application(wsgi.Application):
            """Test application to call from router."""

            def __call__(self, environ, start_response):
                start_response("200", [])
                return ['Router result']

        class Router(wsgi.Router):
            """Test router."""

            def __init__(self):
                mapper = routes.Mapper()
                mapper.connect("/test", controller=Application())
                super(Router, self).__init__(mapper)

        result = webob.Request.blank('/test').get_response(Router())
        self.assertEqual(result.body, "Router result")
        result = webob.Request.blank('/bad').get_response(Router())
        self.assertNotEqual(result.body, "Router result")

 

本身能夠print打樁，瞭解有疑惑的代碼。 而後運行。

$ pyunittest nova/tests/unit/api/test_wsgi.py

 

查看wsgi的實現。

$ git grep -n3 "class Router"
--
nova/wsgi.py-440-        print()
nova/wsgi.py-441-
nova/wsgi.py-442-
nova/wsgi.py:443:class Router(object):
nova/wsgi.py-444-    """WSGI middleware that maps incoming requests to WSGI apps."""
nova/wsgi.py-445-
nova/wsgi.py-446-    def __init__(self, mapper):

 

class Router(object):
    """WSGI middleware that maps incoming requests to WSGI apps."""

    def __init__(self, mapper):
        """Create a router for the given routes.Mapper.

        Each route in `mapper` must specify a 'controller', which is a
        WSGI app to call.  You'll probably want to specify an 'action' as
        well and have your controller be an object that can route
        the request to the action-specific method.

        Examples:
          mapper = routes.Mapper()
          sc = ServerController()

          # Explicit mapping of one route to a controller+action
          mapper.connect(None, '/svrlist', controller=sc, action='list')

          # Actions are all implicitly defined
          mapper.resource('server', 'servers', controller=sc)
          # Pointing to an arbitrary WSGI app.  You can specify the
          # {path_info:.*} parameter so the target app can be handed just that
          # section of the URL.
          mapper.connect(None, '/v1.0/{path_info:.*}', controller=BlogApp())

        """
        self.map = mapper
        self._router = routes.middleware.RoutesMiddleware(self._dispatch,
                                                          self.map)

    @webob.dec.wsgify(RequestClass=Request)
    def __call__(self, req):
        """Route the incoming request to a controller based on self.map.

        If no match, return a 404.

        """
        return self._router

    @staticmethod
    @webob.dec.wsgify(RequestClass=Request)
    def _dispatch(req):
        """Dispatch the request to the appropriate controller.

        Called by self._router after matching the incoming request to a route
        and putting the information into req.environ.  Either returns 404
        or the routed WSGI app's response.

        """
        match = req.environ['wsgiorg.routing_args'][1]
        if not match:
            return webob.exc.HTTPNotFound()
        app = match['controller']
        return app

routes.middleware.RoutesMiddleware的幫助文檔。

Constructor information:
 Definition:routes.middleware.RoutesMiddleware(self, wsgi_app, mapper, use_method_override=True, path_info=True, singleton=True)
 Docstring:
    Create a Route middleware object
    
    Using the use_method_override keyword will require Paste to be
    installed, and your application should use Paste's WSGIRequest
    object as it will properly handle POST issues with wsgi.input
    should Routes check it.
    
    If path_info is True, then should a route var contain
    path_info, the SCRIPT_NAME and PATH_INFO will be altered
    accordingly. This should be used with routes like:
    
    .. code-block:: python
    
        map.connect('blog/*path_info', controller='blog', path_info='')

 

文檔中明確說明： use Paste's WSGIRequest

nova/wsgi.py 實現了load app功能， 從

class Loader(object):
    """Used to load WSGI applications from paste configurations."""

    def __init__(self, config_path=None):
        """Initialize the loader, and attempt to find the config.

        :param config_path: Full or relative path to the paste config.
        :returns: None

        """
        self.config_path = None

        config_path = config_path or CONF.api_paste_config
        if not os.path.isabs(config_path):
            self.config_path = CONF.find_file(config_path)
        elif os.path.exists(config_path):
            self.config_path = config_path

        if not self.config_path:
            raise exception.ConfigNotFound(path=config_path)

    def load_app(self, name):
        """Return the paste URLMap wrapped WSGI application.

        :param name: Name of the application to load.
        :returns: Paste URLMap object wrapping the requested application.
        :raises: `nova.exception.PasteAppNotFound`

        """
        try:
            LOG.debug("Loading app %(name)s from %(path)s",
                      {'name': name, 'path': self.config_path})
            return deploy.loadapp("config:%s" % self.config_path, name=name)
        except LookupError:
            LOG.exception(_LE("Couldn't lookup app: %s"), name)
            raise exception.PasteAppNotFound(name=name, path=self.config_path)

 

至於  server 怎麼加載wsgi的，請參考：

nova/tests/unit/test_wsgi.py

$ pyunittest nova/tests/unit/test_wsgi.TestWSGIServerWithSSL.test_ssl_server

 

最終urlmap的事情交給 paste

請參考：

http://blog.csdn.net/sonicatnoc/article/details/6539716

一如下純拷貝：

首先python paste是一個WSGI工具包，在WSGI的基礎上包裝了幾層，讓應用管理和實現變得方便。說實話，Python Paste的文檔作的真差勁！加之python代碼可讀性原本就不怎麼滴，真費勁。

 

paste.deploy關鍵部分留個抓印：

1）python paste.deploy不能只裝個paste.deploy包就能夠工做了，還須要paste.script包

2）python paste.deploy中loadapp給的路徑可用os.path.abspath(配置文件相對路徑)獲得配置文件的絕對路徑，不然報找不到relative_to path...沒搞明白怎麼回事，目前不重要，放過。

3）python paste.deploy中filter,filter_factory,app,app_factory的規範在文檔中都沒怎麼寫清楚，我來給你補上吧：

- app是一個callable object，接受的參數(environ,start_response)，這是paste系統交給application的，符合

WSGI規範的參數. app須要完成的任務是響應envrion中的請求，準備好響應頭和消息體，而後交給start_response處理，並返回響應消息體。

- filter是一個callable object，其惟一參數是(app)，這是WSGI的application對象，見（1），filter須要完成的工做是將application包 裝成另外一個application（「過濾」），並返回這個包裝後的application。

- app_factory是一個callable object，其接受的參數是一些關於application的配置信息：(global_conf,**kwargs)，global_conf是在 ini文件中default section中定義的一系列key-value對，而**kwargs，即一些本地配置，是在ini文件中，app:xxx section中定義的一 系列key-value對。app_factory返回值是一個application對象

- filter_factory是一個callable object，其接受的參數是一系列關於filter的配置信息：(global_conf,**kwargs)，global_conf是在ini文件 中default section中定義的一系列key-value對，而**kwargs，即一些本地配置，是在ini文件中，filter:xxx section中定 義的一系列key-value對。filter_factory返回一個filter對象

 

給個例子：

pastedeploylab.ini:

[DEFAULT]  
key1=value1  
key2=value2  
key3=values  
[composite:pdl]  
use=egg:Paste#urlmap  
/:root  
/calc:calc  
[pipeline:root]  
pipeline = logrequest showversion  
[pipeline:calc]  
pipeline = logrequest calculator  
[filter:logrequest]  
username = root  
password = root123  
paste.filter_factory = pastedeploylab:LogFilter.factory  
[app:showversion]  
version = 1.0.0  
paste.app_factory = pastedeploylab:ShowVersion.factory  
[app:calculator]  
description = This is an "+-*/" Calculator   
paste.app_factory = pastedeploylab:Calculator.factory 

View Code

pastedeploylab.py

''''' 
Created on 2011-6-12 
@author: Sonic 
'''  
import os  
import webob  
from webob import Request  
from webob import Response  
from paste.deploy import loadapp  
from wsgiref.simple_server import make_server  
#Filter  
class LogFilter():  
    def __init__(self,app):  
        self.app = app  
        pass  
    def __call__(self,environ,start_response):  
        print "filter:LogFilter is called."  
        return self.app(environ,start_response)  
    @classmethod  
    def factory(cls, global_conf, **kwargs):  
        print "in LogFilter.factory", global_conf, kwargs  
        return LogFilter  
class ShowVersion():  
    def __init__(self):  
        pass  
    def __call__(self,environ,start_response):  
        start_response("200 OK",[("Content-type", "text/plain")])  
        return ["Paste Deploy LAB: Version = 1.0.0",]  
    @classmethod  
    def factory(cls,global_conf,**kwargs):  
        print "in ShowVersion.factory", global_conf, kwargs  
        return ShowVersion()  
class Calculator():  
    def __init__(self):  
        pass  
      
    def __call__(self,environ,start_response):  
        req = Request(environ)  
        res = Response()  
        res.status = "200 OK"  
        res.content_type = "text/plain"  
        # get operands  
        operator = req.GET.get("operator", None)  
        operand1 = req.GET.get("operand1", None)  
        operand2 = req.GET.get("operand2", None)  
        print req.GET  
        opnd1 = int(operand1)  
        opnd2 = int(operand2)  
        if operator == u'plus':  
            opnd1 = opnd1 + opnd2  
        elif operator == u'minus':  
            opnd1 = opnd1 - opnd2  
        elif operator == u'star':  
            opnd1 = opnd1 * opnd2  
        elif operator == u'slash':  
            opnd1 = opnd1 / opnd2  
        res.body = "%s /nRESULT= %d" % (str(req.GET) , opnd1)  
        return res(environ,start_response)  
    @classmethod  
    def factory(cls,global_conf,**kwargs):  
        print "in Calculator.factory", global_conf, kwargs  
        return Calculator()  

if __name__ == '__main__':  
    configfile="pastedeploylab.ini"  
    appname="pdl"  
    wsgi_app = loadapp("config:%s" % os.path.abspath(configfile), appname)  
    server = make_server('localhost',8088,wsgi_app)  
    server.serve_forever()  
    pass  

View Code

 

使用：

http://127.0.0.1:8080/

輸出：

Paste Deploy LAB: Version = 1.0.0

http://127.0.0.1:8080/calc?operator=plus&operand1=12&operand2=23

輸出：

UnicodeMultiDict([('operator', u'plus'), ('operand1', u'12'), ('operand2', u'23')])

RESULT= 35

====================================================

進一步猜想filter的使用過程：在paste deploy庫中應該有相似這樣的一段代碼對application進行重組包裝：

#

# 假設在ini文件中, 某條pipeline的順序是filter1, filter2, filter3

# app, 那麼，最終運行的app_real是這樣組織的：

#



app_real = filter1(filter2(filter3(app)))



# 在app真正被調用的過程當中，filter1.__call__(environ,start_response)被首先調用，若某種檢查未經過，filter1作出反應；不然交給filter2__call__(environ,start_response)進一步處理，若某種檢查未經過，filter2作出反應，中斷鏈條，不然交給filter3.__call__(environ,start_response)處理，若filter3的某種檢查都經過了，最後交給app.__call__(environ,start_response)進行處理。

 

看到令賢寫了一篇更簡單的。

http://blog.csdn.net/lynn_kong/article/details/8818005