第二百六十一節，Tornado框架模板引擎本質

時間 2019-11-15

原文原文鏈接

Tornado框架模板引擎本質javascript

只須要了解一下便可css

本篇就來詳細的剖析模板處理的整個過程。html

上圖是返回給用戶一個html文件的整個流程，較以前的Demo多了綠色流線的步驟，其實就是把【self.write('hello world')】變成了【self.render('main.html')】，對於全部的綠色流線只作了五件事：java

使用內置的open函數讀取Html文件中的內容
根據模板語言的標籤分割Html文件的內容，例如：{{}} 或 {%%}
將分割後的部分數據塊格式化成特殊的字符串（表達式）
經過python的內置函數執行字符串表達式，即：將html文件的內容和嵌套的數據整合
將數據返回給請求客戶端

因此，若是要返回給客戶端對於一個html文件來講，根據上述的5個階段其內容的變化過程應該是這樣：python

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.render("main.html",**{'data':['11','22','33'],'title':'main'})

[main.html]
<!DOCTYPE html>
<html>
<head lang="en">
    <meta charset="UTF-8">
    <title></title>
</head>
<body>
    <h1>{{title}}</h1>

    {% for item in data %}
        <h3>{{item}}</h3>
    {% end %}
</body>
</html>

<!DOCTYPE html>
<html>
<head lang="en">
    <meta charset="UTF-8">
    <title></title>
</head>
<body>
    <h1>{{title}}</h1>

    {% for item in data %}
        <h3>{{item}}</h3>
    {% end %}
</body>
</html>

第1塊：'<!DOCTYPE html><html><head lang="en"><meta charset="UTF-8"><title></title></head><h1>'
第2塊：'title'
第3塊：'</h1> \n\n'
第4塊：'for item in data'
    第4.1塊：'\n <h3>'
    第4.2塊：'item'
    第4.3塊：'</h3> \n'
第五塊：'</body>'

'def _execute():
    _buffer = []
    _buffer.append(\\'<!DOCTYPE html>\\n<html>\\n<head lang="en">\\n<meta charset="UTF-8">\\n<title></title>\\n</head>\\n<body>\\n<h1>\\')
    _tmp = title
    if isinstance(_tmp, str): _buffer.append(_tmp)
    elif isinstance(_tmp, unicode): _buffer.append(_tmp.encode(\\'utf-8\\'))
    else: _buffer.append(str(_tmp))
    _buffer.append(\\'</h1>\\n\\')
    for item in data:
        _buffer.append(\\'\\n<h3>\\')
        _tmp = item
        if isinstance(_tmp, str): _buffer.append(_tmp)
        elif isinstance(_tmp, unicode): _buffer.append(_tmp.encode(\\'utf-8\\'))
        else: _buffer.append(str(_tmp))
        _buffer.append(\\'</h3>\\n\\')
    _buffer.append(\\'\\n</body>\\n</html>\\')
    return \\'\\'.join(_buffer)
'

RequestHandler的render方法

此段代碼主要有三項任務：web

獲取Html文件內容並把數據（程序數據或框架自帶數據）嵌套在內容中的指定標籤中（本篇主題）
執行ui_modules，再次在html中插入內容，例：head，js文件、js內容、css文件、css內容和body
內部調用客戶端socket，將處理請求後的數據返回給請求客戶端

class RequestHandler(object):
 
    def render(self, template_name, **kwargs):
        #根據Html文件名稱獲取文件內容並把參數kwargs嵌入到內容的指定標籤內
        html = self.render_string(template_name, **kwargs)
 
        #執行ui_modules，再在html的內容中插入head，js文件、js內容、css文件、css內容和body信息。
        js_embed = []
        js_files = []
        css_embed = []
        css_files = []
        html_heads = []
        html_bodies = []
        for module in getattr(self, "_active_modules", {}).itervalues():
            embed_part = module.embedded_javascript()
            if embed_part: js_embed.append(_utf8(embed_part))
            file_part = module.javascript_files()
            if file_part:
                if isinstance(file_part, basestring):
                    js_files.append(file_part)
                else:
                    js_files.extend(file_part)
            embed_part = module.embedded_css()
            if embed_part: css_embed.append(_utf8(embed_part))
            file_part = module.css_files()
            if file_part:
                if isinstance(file_part, basestring):
                    css_files.append(file_part)
                else:
                    css_files.extend(file_part)
            head_part = module.html_head()
            if head_part: html_heads.append(_utf8(head_part))
            body_part = module.html_body()
            if body_part: html_bodies.append(_utf8(body_part))
        if js_files:#添加js文件
            # Maintain order of JavaScript files given by modules
            paths = []
            unique_paths = set()
            for path in js_files:
                if not path.startswith("/") and not path.startswith("http:"):
                    path = self.static_url(path)
                if path not in unique_paths:
                    paths.append(path)
                    unique_paths.add(path)
            js = ''.join('<script src="' + escape.xhtml_escape(p) +
                         '" type="text/javascript"></script>'
                         for p in paths)
            sloc = html.rindex('</body>')
            html = html[:sloc] + js + '\n' + html[sloc:]
        if js_embed:#添加js內容
            js = '<script type="text/javascript">\n//<![CDATA[\n' + \
                '\n'.join(js_embed) + '\n//]]>\n</script>'
            sloc = html.rindex('</body>')
            html = html[:sloc] + js + '\n' + html[sloc:]
        if css_files:#添加css文件
            paths = []
            unique_paths = set()
            for path in css_files:
                if not path.startswith("/") and not path.startswith("http:"):
                    path = self.static_url(path)
                if path not in unique_paths:
                    paths.append(path)
                    unique_paths.add(path)
            css = ''.join('<link href="' + escape.xhtml_escape(p) + '" '
                          'type="text/css" rel="stylesheet"/>'
                          for p in paths)
            hloc = html.index('</head>')
            html = html[:hloc] + css + '\n' + html[hloc:]
        if css_embed:#添加css內容
            css = '<style type="text/css">\n' + '\n'.join(css_embed) + \
                '\n</style>'
            hloc = html.index('</head>')
            html = html[:hloc] + css + '\n' + html[hloc:]
        if html_heads:#添加html的header
            hloc = html.index('</head>')
            html = html[:hloc] + ''.join(html_heads) + '\n' + html[hloc:]
        if html_bodies:#添加html的body
            hloc = html.index('</body>')
            html = html[:hloc] + ''.join(html_bodies) + '\n' + html[hloc:]
             
        #把處理後的信息響應給客戶端
        self.finish(html)

對於上述三項任務，第一項是模板語言的重中之重，讀取html文件並將數據嵌套到指定標籤中，如下的步驟用於剖析整個過程（詳情見下文）；第二項是對返會給用戶內容的補充，也就是在第一項處理完成以後，利用ui_modules再次在html中插入內容（head，js文件、js內容、css文件、css內容和body）；第三項是經過socket將內容響應給客戶端（見上篇）。express

對於ui_modules，每個ui_module其實就是一個類，一旦註冊並激活了該ui_module，tornado便會自動執行其中的方法：embedded_javascript、javascript_files、embedded_css、css_files、html_head、html_body和render ，從而實現對html內容的補充。（執行過程見上述代碼）json

自定義UI Modulesapp

此處是一個完整的建立 --> 註冊 --> 激活的Demo框架

目錄結構：

　　├── index.py
　　├── static
　　└── views
　　　　　└── index.html

#!/usr/bin/env python
# -*- coding:utf-8 -*-


import tornado.ioloop
import tornado.web


class CustomModule(tornado.web.UIModule):
    def embedded_javascript(self):
        return 'embedded_javascript'

    def javascript_files(self):
        return 'javascript_files'

    def embedded_css(self):
        return 'embedded_css'

    def css_files(self):
        return 'css_files'

    def html_head(self):
        return 'html_head'

    def html_body(self):
        return 'html_body'

    def render(self):
        return 'render'


class MainHandler(tornado.web.RequestHandler):

    def get(self):
        self.render('index.html')

settings = {
    'static_path': 'static',
    "template_path": 'views',
    "ui_modules": {'Foo': CustomModule},
    }

application = tornado.web.Application([(r"/", MainHandler), ], **settings)

if __name__ == "__main__":
    application.listen(8888)
    tornado.ioloop.IOLoop.instance().start()

<!DOCTYPE html>
<html>
<head lang="en">
    <meta charset="UTF-8">
    <title></title>
</head>
<body>

    <hr>
    {% module Foo() %}
    <hr>

</body>
</html>

<!DOCTYPE html>
<html>
<head lang="en">
    <meta charset="UTF-8">
    <title></title>
    <!-- css_files -->
    <link href="/static/css_files" type="text/css" rel="stylesheet">
    
    <!-- embedded_css -->
    <style type="text/css">
        embedded_css
    </style>

</head>
<body>
    
    <!-- html_head -->
    html_head



    <hr>
    <!-- redner -->
    render
    <hr>


    <!-- javascript_files -->
    <script src="/static/javascript_files" type="text/javascript"></script>


    <!-- embedded_javascript -->
    <script type="text/javascript">
    //<![CDATA[
        embedded_javascript
    //]]>
    </script>

    <!-- html_body -->
    html_body

</body>
</html>

RequestHandler的render_string方法

該方法是本篇的重中之重，它負責去處理Html模板並返回最終結果，【概述】中提到的5件事中前四件都是此方法來完成的，即：

建立Loader對象，並執行load方法
-- 經過open函數打開html文件並讀取內容，並將內容做爲參數又建立一個 Template 對象
-- 當執行Template的 __init__ 方法時，根據模板語言的標籤 {{}}、{%%}等分割並html文件，最後生成一個字符串表示的函數
獲取全部要嵌入到html模板中的變量，包括：用戶返回和框架默認
執行Template對象的generate方法
-- 編譯字符串表示的函數，並將用戶定義的值和框架默認的值做爲全局變量
-- 執行被編譯的函數獲取被嵌套了數據的內容，而後將內容返回（用於響應給請求客戶端）

class RequestHandler(object):
 
    def render_string(self, template_name, **kwargs):
         
        #獲取配置文件中指定的模板文件夾路徑，即：template_path ＝ 'views'
        template_path = self.get_template_path()
 
        #若是沒有配置模板文件的路徑，則默認去啓動程序所在的目錄去找
        if not template_path:
            frame = sys._getframe(0)
            web_file = frame.f_code.co_filename
            while frame.f_code.co_filename == web_file:
                frame = frame.f_back
            template_path = os.path.dirname(frame.f_code.co_filename)
        if not getattr(RequestHandler, "_templates", None):
            RequestHandler._templates = {}
         
        #建立Loader對象，第一次建立後，會將該值保存在RequestHandler的靜態字段_template_loaders中
        if template_path not in RequestHandler._templates:
            loader = self.application.settings.get("template_loader") or\
              template.Loader(template_path)
            RequestHandler._templates[template_path] = loader
 
        #執行Loader對象的load方法，該方法內部執行執行Loader的_create_template方法
        #在_create_template方法內部使用open方法會打開html文件並讀取html的內容，而後將其做爲參數來建立一個Template對象
        #Template的構造方法被執行時，內部解析html文件的內容，並根據內部的 {{}} {%%}標籤對內容進行分割，最後生成一個字符串類表示的函數並保存在self.code字段中
        t = RequestHandler._templates[template_path].load(template_name)
         
        #獲取全部要嵌入到html中的值和框架默認提供的值
        args = dict(
            handler=self,
            request=self.request,
            current_user=self.current_user,
            locale=self.locale,
            _=self.locale.translate,
            static_url=self.static_url,
            xsrf_form_html=self.xsrf_form_html,
            reverse_url=self.application.reverse_url
        )
        args.update(self.ui)
        args.update(kwargs)
 
        #執行Template的generate方法，編譯字符串表示的函數並將namespace中的全部key，value設置成全局變量，而後執行該函數。從而將值嵌套進html並返回。
        return t.generate(**args)

class Loader(object):
    """A template loader that loads from a single root directory.

    You must use a template loader to use template constructs like
    {% extends %} and {% include %}. Loader caches all templates after
    they are loaded the first time.
    """
    def __init__(self, root_directory):
        self.root = os.path.abspath(root_directory)
        self.templates = {}

class Loader(object):
    def load(self, name, parent_path=None):
        name = self.resolve_path(name, parent_path=parent_path)
        if name not in self.templates:
            path = os.path.join(self.root, name)
            f = open(path, "r")
            #讀取html文件的內容
            #建立Template對象
            #name是文件名
            self.templates[name] = Template(f.read(), name=name, loader=self)
            f.close()
        return self.templates[name]

class Template(object):

    def __init__(self, template_string, name="<string>", loader=None,compress_whitespace=None):
        # template_string是Html文件的內容

        self.name = name
        if compress_whitespace is None:
            compress_whitespace = name.endswith(".html") or name.endswith(".js")
        
        #將內容封裝到_TemplateReader對象中，用於以後根據模板語言的標籤分割html文件
        reader = _TemplateReader(name, template_string)
        
        #分割html文件成爲一個一個的對象
        #執行_parse方法，將html文件分割成_ChunkList對象
        self.file = _File(_parse(reader))
        
        #將html內容格式化成字符串表示的函數
        self.code = self._generate_python(loader, compress_whitespace)
        
        try:
            #將字符串表示的函數編譯成函數
            self.compiled = compile(self.code, self.name, "exec")
            
        except:
            formatted_code = _format_code(self.code).rstrip()
            logging.error("%s code:\n%s", self.name, formatted_code)
            raise

class Template(object):
    def generate(self, **kwargs):
        """Generate this template with the given arguments."""
        namespace = {
            "escape": escape.xhtml_escape,
            "xhtml_escape": escape.xhtml_escape,
            "url_escape": escape.url_escape,
            "json_encode": escape.json_encode,
            "squeeze": escape.squeeze,
            "linkify": escape.linkify,
            "datetime": datetime,
        }

        #建立變量環境並執行函數，詳細Demo見上一篇博文
        namespace.update(kwargs)
        exec self.compiled in namespace
        execute = namespace["_execute"]

        try:
            #執行編譯好的字符串格式的函數，獲取嵌套了值的html文件
            return execute()
        except:
            formatted_code = _format_code(self.code).rstrip()
            logging.error("%s code:\n%s", self.name, formatted_code)
            raise

其中涉及的類有：

class _TemplateReader(object):
    def __init__(self, name, text):
        self.name = name
        self.text = text
        self.line = 0
        self.pos = 0

    def find(self, needle, start=0, end=None):
        assert start >= 0, start
        pos = self.pos
        start += pos
        if end is None:
            index = self.text.find(needle, start)
        else:
            end += pos
            assert end >= start
            index = self.text.find(needle, start, end)
        if index != -1:
            index -= pos
        return index

    def consume(self, count=None):
        if count is None:
            count = len(self.text) - self.pos
        newpos = self.pos + count
        self.line += self.text.count("\n", self.pos, newpos)
        s = self.text[self.pos:newpos]
        self.pos = newpos
        return s

    def remaining(self):
        return len(self.text) - self.pos

    def __len__(self):
        return self.remaining()

    def __getitem__(self, key):
        if type(key) is slice:
            size = len(self)
            start, stop, step = key.indices(size)
            if start is None: start = self.pos
            else: start += self.pos
            if stop is not None: stop += self.pos
            return self.text[slice(start, stop, step)]
        elif key < 0:
            return self.text[key]
        else:
            return self.text[self.pos + key]

    def __str__(self):
        return self.text[self.pos:]

class _ChunkList(_Node):
    def __init__(self, chunks):
        self.chunks = chunks

    def generate(self, writer):
        for chunk in self.chunks:
            chunk.generate(writer)

    def each_child(self):
        return self.chunks

def _parse(reader, in_block=None):
    
    #默認建立一個內容爲空列表的_ChunkList對象
    body = _ChunkList([])
    
    # 將html塊添加到 body.chunks 列表中
    while True:
        # Find next template directive
        curly = 0
        while True:
            curly = reader.find("{", curly)
            if curly == -1 or curly + 1 == reader.remaining():
                # EOF
                if in_block:
                    raise ParseError("Missing {%% end %%} block for %s" %in_block)
                body.chunks.append(_Text(reader.consume()))
                return body
            # If the first curly brace is not the start of a special token,
            # start searching from the character after it
            if reader[curly + 1] not in ("{", "%"):
                curly += 1
                continue
            # When there are more than 2 curlies in a row, use the
            # innermost ones.  This is useful when generating languages
            # like latex where curlies are also meaningful
            if (curly + 2 < reader.remaining() and
                reader[curly + 1] == '{' and reader[curly + 2] == '{'):
                curly += 1
                continue
            break

        # Append any text before the special token
        if curly > 0:
            body.chunks.append(_Text(reader.consume(curly)))

        start_brace = reader.consume(2)
        line = reader.line

        # Expression
        if start_brace == "{{":
            end = reader.find("}}")
            if end == -1 or reader.find("\n", 0, end) != -1:
                raise ParseError("Missing end expression }} on line %d" % line)
            contents = reader.consume(end).strip()
            reader.consume(2)
            if not contents:
                raise ParseError("Empty expression on line %d" % line)
            body.chunks.append(_Expression(contents))
            continue

        # Block
        assert start_brace == "{%", start_brace
        end = reader.find("%}")
        if end == -1 or reader.find("\n", 0, end) != -1:
            raise ParseError("Missing end block %%} on line %d" % line)
        contents = reader.consume(end).strip()
        reader.consume(2)
        if not contents:
            raise ParseError("Empty block tag ({%% %%}) on line %d" % line)

        operator, space, suffix = contents.partition(" ")
        suffix = suffix.strip()

        # Intermediate ("else", "elif", etc) blocks
        intermediate_blocks = {
            "else": set(["if", "for", "while"]),
            "elif": set(["if"]),
            "except": set(["try"]),
            "finally": set(["try"]),
        }
        allowed_parents = intermediate_blocks.get(operator)
        if allowed_parents is not None:
            if not in_block:
                raise ParseError("%s outside %s block" %
                            (operator, allowed_parents))
            if in_block not in allowed_parents:
                raise ParseError("%s block cannot be attached to %s block" % (operator, in_block))
            body.chunks.append(_IntermediateControlBlock(contents))
            continue

        # End tag
        elif operator == "end":
            if not in_block:
                raise ParseError("Extra {%% end %%} block on line %d" % line)
            return body

        elif operator in ("extends", "include", "set", "import", "from",
                          "comment"):
            if operator == "comment":
                continue
            if operator == "extends":
                suffix = suffix.strip('"').strip("'")
                if not suffix:
                    raise ParseError("extends missing file path on line %d" % line)
                block = _ExtendsBlock(suffix)
            elif operator in ("import", "from"):
                if not suffix:
                    raise ParseError("import missing statement on line %d" % line)
                block = _Statement(contents)
            elif operator == "include":
                suffix = suffix.strip('"').strip("'")
                if not suffix:
                    raise ParseError("include missing file path on line %d" % line)
                block = _IncludeBlock(suffix, reader)
            elif operator == "set":
                if not suffix:
                    raise ParseError("set missing statement on line %d" % line)
                block = _Statement(suffix)
            body.chunks.append(block)
            continue

        elif operator in ("apply", "block", "try", "if", "for", "while"):
            # parse inner body recursively
            block_body = _parse(reader, operator)
            if operator == "apply":
                if not suffix:
                    raise ParseError("apply missing method name on line %d" % line)
                block = _ApplyBlock(suffix, block_body)
            elif operator == "block":
                if not suffix:
                    raise ParseError("block missing name on line %d" % line)
                block = _NamedBlock(suffix, block_body)
            else:
                block = _ControlBlock(contents, block_body)
            body.chunks.append(block)
            continue

        else:
            raise ParseError("unknown operator: %r" % operator)

class Template(object):

    def _generate_python(self, loader, compress_whitespace):
        buffer = cStringIO.StringIO()
        try:
            named_blocks = {}
            ancestors = self._get_ancestors(loader)
            ancestors.reverse()
            for ancestor in ancestors:
                ancestor.find_named_blocks(loader, named_blocks)
            self.file.find_named_blocks(loader, named_blocks)
            writer = _CodeWriter(buffer, named_blocks, loader, self,
                                 compress_whitespace)
            ancestors[0].generate(writer)
            return buffer.getvalue()
        finally:
            buffer.close()

so，上述整個過程其實就是將一個html轉換成一個函數，併爲該函數提供全局變量，而後執行該函數！！

結束語

上述就是對於模板語言的整個流程，其本質就是處理html文件內容將html文件內容轉換成函數，而後爲該函數提供全局變量環境（即：咱們想要嵌套進html中的值和框架自帶的值），再以後執行該函數從而獲取處處理後的結果，再再以後則執行UI_Modules繼續豐富返回結果，例如：添加js文件、添加js內容塊、添加css文件、添加css內容塊、在body內容第一行插入數據、在body內容最後同樣插入數據，最終，經過soekct客戶端對象將處理以後的返回結果（字符串）響應給請求用戶。