因sentry使用的是WSGI協議, 所以此文先簡述此協議;html
而後講解sentry中是如何處理api請求, 以及對應的源碼講解.java
首先弄清下面幾個概念: WSGI: 一種通訊協議. 全稱是Web Server Gateway Interface
,python
模塊,框架,API
或者任何軟件,只是一種規範,描述web server
如何與web application
通訊的規範。server
和application
的規範在PEP 3333中有具體描述。要實現WSGI協議,必須同時實現web server和web application,當前運行在WSGI
協議之上的web
框架有Bottle
, Flask
, Django
。 **uwsgi:**與WSGI
同樣是一種通訊協議,是uWSGI
服務器的獨佔協議,用於定義傳輸信息的類型(type of information
),每個uwsgi packet
前4byte
爲傳輸信息類型的描述,與WSGI協議是兩種東西,聽說該協議是fcgi
協議的10倍快。 **uWSGI:**是一個web
服務器,實現了WSGI
協議、uwsgi
協議、http
協議等。python
WSGI
協議主要包括server
和application
兩部分:nginx
WSGI server
負責從客戶端接收請求,將request
轉發給application
,將application
返回的response
返回給客戶端;WSGI application
接收由server
轉發的request
,處理請求,並將處理結果返回給server
。application
中能夠包括多個棧式的中間件(middlewares
),這些中間件須要同時實現server與application,所以能夠在WSGI服務器與WSGI應用之間起調節做用:對服務器來講,中間件扮演應用程序,對應用程序來講,中間件扮演服務器。WSGI
協議實際上是定義了一種server
與application
解耦的規範,便可以有多個實現WSGI server
的服務器,也能夠有多個實現WSGI application
的框架,那麼就能夠選擇任意的server
和application
組合實現本身的web
應用。例如uWSGI
和Gunicorn
都是實現了WSGI server
協議的服務器,Django
,Flask
是實現了WSGI application
協議的web
框架,能夠根據項目實際狀況搭配使用。git
wsgi.py django項目攜帶的一個wsgi接口文件 若是項目名叫app的話,此文件就位於[app/app/wsgi.py]github
WSGI的工做原理分爲服務器層和應用程序層:web
參考文檔:django
WSGI&uwsgi: www.jianshu.com/p/679dee0a4…json
WSGI工做原理及實現:geocld.github.io/2017/08/14/…api
server具體成 uwsgi, application具體成django wsgi application (是一個可調用的方法 or class or 函數)
def simple_app(environ, start_response):
"""Simplest possible application object"""
status = '200 OK'
response_headers = [('Content-type', 'text/html')]
start_response(status, response_headers)
return ['Hello World']
複製代碼
class simple_app_class(object):
def __call__(self, environ, start_response):
.....
複製代碼
WSGI應用的規定必須有如下兩個參數:
environment
server
,同時返回響應正文(response body
),響應正文是可迭代的、幷包含了多個字符串App.py
def application(env, start_response):
start_response('200 OK', [('Content-Type', 'text/html'), ('X-Coder', 'Cooffeeli')])
return ['<h1>你好!!世界</h1>']
複製代碼
wsgi_server.py(python 的 wsgiref 庫運行一個 WSGI 服務器)
from wsgiref.simple_server import make_server
from app import application
# 啓動 WSGI 服務器
httpd = make_server (
'localhost',
9000,
application # 這裏指定咱們的 application object)
)
# 開始處理請求
httpd.handle_request()
複製代碼
wsgi_server.py(本身實現複雜的)
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import socket
import sys
import StringIO
from app import application
from datetime import datetime
class WSGIServer(object):
def __init__(self, server_address):
"""初始構造函數, 建立監聽socket"""
self.listen_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.listen_sock.bind(server_address)
self.listen_sock.listen(5)
(host, port) = self.listen_sock.getsockname()
self.server_port = port
self.server_name = socket.getfqdn(host)
def set_application(self, application):
"""設置wsgi application, 供server 調用"""
self.application = application
def get_environ(self):
"""構造WSGI環境變量,傳給application的env參數"""
self.env = {
'wsgi.version': (1, 0),
'wsgi.url_scheme': 'http',
'wsgi.errors': sys.stderr,
'wsgi.multithread': False,
'wsgi.run_once': False,
'REQUEST_METHOD': self.request_method,
'PATH_INFO': self.request_path,
'SERVER_NAME': self.server_name,
'SERVER_PORT': str(self.server_port),
'wsgi.input': StringIO.StringIO(self.request_data),
}
return self.env
def start_response(self, http_status, http_headers):
"""構造WSGI響應, 傳給application的start_response"""
self.http_status = http_status
self.http_headers = dict(http_headers)
headers = {
'Date': datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT'),
'Server': 'WSGIServer 1.0'
}
self.http_headers.update(headers)
def parse_request(self, text):
"""獲取http頭信息,用於構造env參數"""
request_line = text.splitlines()[0]
request_info = request_line.split(' ')
(self.request_method,
self.request_path,
self.request_version) = request_info
def get_http_response(self, response_data):
"""完成response 內容"""
res = 'HTTP/1.1 {status} \r\n'.format(status=self.http_status)
for header in self.http_headers.items():
res += '{0}: {1} \r\n'.format(*header)
res += '\r\n'
res_body = ''
for val in response_data:
res_body += val
res += res_body
return res
def handle_request(self):
"""處理請求"""
# 初始版本,只接受一個請求
conn, addr = self.listen_sock.accept()
# 獲取http 請求的request內容
self.request_data = conn.recv(1024)
self.parse_request(self.request_data)
# 構造調用application須要的兩個參數 env, start_response
env = self.get_environ()
start_response = self.start_response
# 調用application, 並獲取須要返回的http response內容
response_data = self.application(env, start_response)
# 獲取完整http response header 和 body, 經過socket的sendall返回到客戶端
res = self.get_http_response(response_data)
conn.sendall(res)
# 腳本運行完畢也會結束
conn.close()
def make_server(server_address, application):
"""建立WSGI Server 負責監聽端口,接受請求"""
wsgi_server = WSGIServer(server_address)
wsgi_server.set_application(application)
return wsgi_server
SERVER_ADDRESS = (HOST, PORT) = '', 8124
wsgi_server = make_server(SERVER_ADDRESS, application)
wsgi_server.handle_request()
複製代碼
命令行參數 直接使用—wokers 或者 -w
環境變量形式 使用UWSGI_ 開頭,而後把全部的參數都大寫 UWSGI_WORKERS
使用xml配置
<uwsgi>
<master>
...
<workers>4</workers>
...
<master/>
</uwsgi>
複製代碼
參考文檔:
uwsgi中文文檔:uwsgi-docs-zh.readthedocs.io/zh_CN/lates…
uwsgi參數講解:mhl.xyz/Python/uwsg…
web 啓動命令: sentry --config . run web [-w 5 等其餘可選命令]
web對應的方法爲: runner/commands/run.py 中的web()方法, 此方法中調用了SentryHTTPServer():
第一步: 調用__init__對options進行了初始化
第二步: 調用此方法中的run()
第一步: 準備環境變量, 將uwsgi的options配置設置到環境變量中(參見: Uwsgi 參數配置, 見源碼分析); 須要注意的是在options中設置了uwsgi協議中的application "sentry.wsgi:application"
第二步: 使用os.execvp直接啓動uwsgi服務(建立一個WSGIServer類的實例)
class SentryHTTPServer(Service):
name = 'http'
def __init__( self, host=None, port=None, debug=False, workers=None, validate=True, extra_options=None ):
from django.conf import settings
from sentry import options as sentry_options
from sentry.logging import LoggingFormat
if validate:
self.validate_settings()
host = host or settings.SENTRY_WEB_HOST
port = port or settings.SENTRY_WEB_PORT
options = (settings.SENTRY_WEB_OPTIONS or {}).copy()
if extra_options is not None:
for k, v in six.iteritems(extra_options):
options[k] = v
# 此配置是uwsgi的參數, 加載一個WSGI模塊
options.setdefault('module', 'sentry.wsgi:application')
......
# 限制請求體 設置爲0表示沒有限制
options.setdefault('limit-post', 0)
# 後面還有一堆的設置 選項:https://uwsgi-docs-zh.readthedocs.io/zh_CN/latest/Options.html
# 路徑 jex-backend/src/sentry/services/http.py
......
複製代碼
class WSGIHandler(base.BaseHandler):
initLock = Lock()
request_class = WSGIRequest
def __call__(self, environ, start_response):
# 加載中間件 (下面有講解)
if self._request_middleware is None:
with self.initLock:
try:
# Check that middleware is still uninitialized.
if self._request_middleware is None:
self.load_middleware()
except:
# Unload whatever middleware we got
self._request_middleware = None
raise
set_script_prefix(get_script_name(environ))
# 請求處理以前發送信號
signals.request_started.send(sender=self.__class__, environ=environ)
try:
request = self.request_class(environ)
except UnicodeDecodeError:
logger.warning('Bad Request (UnicodeDecodeError)',
exc_info=sys.exc_info(),
extra={'status_code': 400,})
response = http.HttpResponseBadRequest()
else:
# 此方法內部對請求的url進行了解析, 執行中間件找到對應的view(幾種類型的中間件執行順序參見最下面的中間件講解的內容), 此方法爲核心方法
response = self.get_response(request)
response._handler_class = self.__class__
status = '%s %s' % (response.status_code, response.reason_phrase)
response_headers = [(str(k), str(v)) for k, v in response.items()]
for c in response.cookies.values():
response_headers.append((str('Set-Cookie'), str(c.output(header=''))))
# server提供的回調方法,將響應的header和status返回給server
start_response(force_str(status), response_headers)
if getattr(response, 'file_to_stream', None) is not None and environ.get('wsgi.file_wrapper'):
response = environ['wsgi.file_wrapper'](response.file_to_stream)
return response
複製代碼
能夠看出application
的流程包括:
jex-backend/env/lib/python2.7/site-packages/django/core/handlers/base.py class: BaseHandler func:load_middleware()
,以及執行框架相關的操做,設置當前線程腳本前綴,發送請求開始信號;get_response()
方法處理當前請求,該方法的的主要邏輯是經過urlconf
找到對應的view
和callback
,按順序執行各類middleware
和callback
。server
傳入的start_response()
方法將響應header
與status
返回給server
。def get_response(self, request):
"Returns an HttpResponse object for the given HttpRequest"
# Setup default url resolver for this thread, this code is outside
# the try/except so we don't get a spurious "unbound local
# variable" exception in the event an exception is raised before
# resolver is set
urlconf = settings.ROOT_URLCONF
urlresolvers.set_urlconf(urlconf)
resolver = urlresolvers.RegexURLResolver(r'^/', urlconf)
try:
response = None
# Apply request middleware
for middleware_method in self._request_middleware:
response = middleware_method(request)
if response:
break
if response is None:
if hasattr(request, 'urlconf'):
# Reset url resolver with a custom urlconf.
urlconf = request.urlconf
urlresolvers.set_urlconf(urlconf)
resolver = urlresolvers.RegexURLResolver(r'^/', urlconf)
# 若是請求時下面的栗子中的url, 則request.path_info=/api/jd/jone/project/
resolver_match = resolver.resolve(request.path_info)
# resolver_match 爲ResolverMatch對象, 內容爲(func=<function JoneProject at 0x110f17410>, args=(), kwargs={}, url_name='create-new-project', app_name='None', namespace='') 所以callback=JoneProject, callback_args=args, callback_kwargs=kwargs
callback, callback_args, callback_kwargs = resolver_match
request.resolver_match = resolver_match
# Apply view middleware
for middleware_method in self._view_middleware:
response = middleware_method(request, callback, callback_args, callback_kwargs)
if response:
break
if response is None:
wrapped_callback = self.make_view_atomic(callback)
try:
# 真正調用view,一下面的例子爲例 調用的是JoneProject中的Post方法, response爲post方法返回的結果, 此方法須要的參數在request.DATA中
response = wrapped_callback(request, *callback_args, **callback_kwargs)
except Exception as e:
# If the view raised an exception, run it through exception
# middleware, and if the exception middleware returns a
# response, use that. Otherwise, reraise the exception.
for middleware_method in self._exception_middleware:
response = middleware_method(request, e)
if response:
break
if response is None:
raise
# Complain if the view returned None (a common error).
......
複製代碼
在get_response()方法中調用resolver_match = resolver.resolve(request.path_info)
對請求的url進行解析
def resolve(self, path):
tried = []
# 正則匹配, 此處的regex爲get_response中調用urlresolvers.RegexURLResolver(r'^/', urlconf)時傳入的表達式 即: 已/開頭的內容
match = self.regex.search(path)
if match:
# 若是請求時下面的例子, path=/api/jd/jone/project/, 則匹配後new_path=api/jd/jone/project/
new_path = path[match.end():]
for pattern in self.url_patterns:
# 遍歷的是在配置中設置的ROOT_URLCONF文件中的內容urlpatterns 若是配置中含有inclde會遞歸遍歷 若是請求的url沒有找到對應匹配則直接返回失敗, 下面例子中的請求的url,第一次遍歷找到對應的 url(r'^api/jd/', include('sentry.api.jdapi.urls')), pattern.resolve(new_path)再次進行遞歸匹配再次獲得new_path=jone/project/ 而後再到sentry.api.jdapi.urls文件中再次進行匹配知道找到對應的url(r'^jone/project/$', JoneProject.as_view(), name="create-new-project")
try:
sub_match = pattern.resolve(new_path)
except Resolver404 as e:
sub_tried = e.args[0].get('tried')
if sub_tried is not None:
tried.extend([[pattern] + t for t in sub_tried])
else:
tried.append([pattern])
else:
if sub_match:
sub_match_dict = dict(match.groupdict(), **self.default_kwargs)
sub_match_dict.update(sub_match.kwargs)
return ResolverMatch(sub_match.func, sub_match.args, sub_match_dict, sub_match.url_name, self.app_name or sub_match.app_name, [self.namespace] + sub_match.namespaces)
tried.append([pattern])
raise Resolver404({'tried': tried, 'path': new_path})
raise Resolver404({'path' : path})
複製代碼
class JoneProjectSerializer(serializers.Serializer):
project_slug = serializers.CharField(min_length=1, max_length=200, required=True)
organization_slug = serializers.CharField(min_length=1, max_length=200, required=True)
platform = serializers.CharField(required=False, default="java")
members = MembersSerializer(many=True)
class JoneProject(APIView):
permission_classes = (RpcPermission,)
def post(self, request):
serializer = JoneProjectSerializer(data=request.DATA)
error_msg = "參數錯誤"
if serializer.is_valid():
result = serializer.object
try:
project_slug = result['project_slug']
valid_project(project_slug)
project = create_project(project_slug, result['platform'], result['members'], result['organization_slug'])
project_key = get_project_key(project)
if project_key:
return response(project_key)
except Exception as e:
logger.exception("建立項目失敗, params: %s, msg:%s", request.DATA, e)
error_msg = e.message
else:
logger.info("參數錯誤:{}".format(serializer.errors))
return response(error_msg, RESPONSE_ERROR_STATUS)
複製代碼
請求URL: http://127.0.0.1:9000/api/jd/jone/project/ POST
參數:
{
"members":[
{
"email":"pengchang@jd.com",
"erp":"pengchang5"
},
{
"email":"guohuixin@jd.com",
"erp":"guohuixin3"
}
],
"platform":"java",
"organization_slug":"org_test",
"project_slug":"pro_test"
}
複製代碼
resolve解析結果: resolver_match:ResolverMatch(func=<function JoneProject at 0x110f17410>, args=(), kwargs={}, url_name='create-new-project', app_name='None', namespace='')
參考文檔:
WSGI & uwsgi講解: www.jianshu.com/p/679dee0a4…
Django從請求到響應的過程: juejin.im/post/5a6c4c…
當用戶向你的應用發送一個請求的時候,一個 WSGI handler 將會被初始化,它會完成如下工做:
參考文檔:
從請求到響應 django 都作了哪些處理: juejin.im/post/5a6951…
中間件是位於Web服務器端和Web應用之間的,它能夠添加額外的功能;
中間件要麼對來自用戶的數據進行預處理,而後發送給應用;要麼在應用將響應負載返回給用戶以前,對結果數據進行一些最終的調整。通俗一點,在django中,中間可以幫咱們準備好request這個對象,而後應用能夠直接使用request對象獲取到各種數據,也幫咱們將response添加頭部,狀態碼等
建立django項目, 默認會添加中間件 MIDDLEWARE_CLASSES ; 若是有新的能夠在此配置中進行添加
django.contrib.auth.middleware.AuthenticationMiddleware:
def get_user(request):
if not hasattr(request, '_cached_user'):
request._cached_user = auth.get_user(request)
return request._cached_user
class AuthenticationMiddleware(MiddlewareMixin):
def process_request(self, request):
assert hasattr(request, 'session'), (
"The Django authentication middleware requires session middleware "
"to be installed. Edit your MIDDLEWARE%s setting to insert "
"'django.contrib.sessions.middleware.SessionMiddleware' before "
"'django.contrib.auth.middleware.AuthenticationMiddleware'."
) % ("_CLASSES" if settings.MIDDLEWARE is None else "")
request.user = SimpleLazyObject(lambda: get_user(request))
複製代碼
這裏咱們能夠發現 request.user 這個屬性是在 AuthenticationMiddleware 中產生的。這個咱們稍後再說。
這裏咱們能夠發現,這個中間件只有 process_request,說明它只在 request 這一步處理流入和流出 django 應用的數據流。這個中間件會首先驗證會話中間件是否被使用,而後經過調用 get_user 函數來設置用戶。當 WSGI 處理程序迭代 process_request 方法列表的時候,它將會構建這個最終會被傳遞給視圖函數的請求對象,並可以使你引用 request.user。一些中間件沒有 process_request 方法,在這個階段,會被跳過。
process_request 應該返回 None 或者 HTTPResponse 對象。當返回 None 時,WSGI handler 會繼續加載 process_request 裏面的方法,可是後一種狀況會短路處理過程並進入 process_response 循環。
當全部的 process_request 被調用完以後,咱們就會獲得一個將被傳遞給視圖函數的 request 對象。當這個事件發生以前,django 必須解析 url 並決定調用哪個視圖函數。這個過程很是簡單,只須要使用正則匹配便可。settings.py 中有一個 ROOT_URLCONF 鍵來指定根 url.py,在這裏會包含你全部 app 的 urls.py 文件。若是沒有匹配成功,將會拋出一個異常 django.core.urlresolvers.Resolver404, 這是 django.http.HTTP404 的子類。
def get_response(self, request):
"Returns an HttpResponse object for the given HttpRequest"
# Setup default url resolver for this thread, this code is outside
# the try/except so we don't get a spurious "unbound local
# variable" exception in the event an exception is raised before
# resolver is set
urlconf = settings.ROOT_URLCONF
urlresolvers.set_urlconf(urlconf)
resolver = urlresolvers.RegexURLResolver(r'^/', urlconf)
try:
response = None
# Apply request middleware
for middleware_method in self._request_middleware:
response = middleware_method(request)
if response:
break
......
複製代碼
到這一步以後 WSGI handler 知道了調用哪個視圖函數,以及傳遞哪些參數。它會再一次調用中間件列表裏面的方法,此次是_view_middleware 列表。全部 Django 中間件的 process_view 方法將會被這樣聲明:
process_view(request, view_function, view_args, view_kwargs)
複製代碼
和 process_request 同樣,process_view 函數必須返回 None 或者 HTTPResponse 對象,使得 WSGI handler 繼續處理視圖或者’短路’處理流程並返回一個響應。在 CSRF middleware 中存在一個 process_view 的方法。做用是當 CSRF cookies 出現時,process_view 方法將會返回 None, 視圖函數將會繼續的執行。若是不是這樣,請求將會被拒絕,處理流程將會被’短路’,會生成一個錯誤的信息。
一個視圖函數須要知足三個條件:
若是視圖函數拋出一個異常,Handler 將會循環遍歷_exception_middleware 列表,這些方法按照相反的順序執行,從 settings.py 裏面列出來的最後一箇中間件到第一個。若是一個異常被拋出,處理過程將會被短路,其餘的 process_exception 將不會被執行。一般咱們依賴 Djnago's BaseHandler 提供的異常處理程序,可是咱們也可使用自定義的異常處理中間件
在這個階段,咱們獲得了一個 HTTPResponse 對象,這個對象多是 process_view 返回的,也多是視圖函數返回的。如今咱們將循環訪問響應中間件。這是中間件調整數據的最後的機會。執行的順序是從內向外執行。 以 cache middleware 的 process_response 爲例:它依賴於你的 app 裏面的不一樣的狀態(緩存是否打開或者關閉,是否在處理一個數據流),來決定是否緩存你的響應。
參考文檔:
django從請求到響應的過程: juejin.im/post/5a6c4c…