nginx+uwsgi+django+supervisor+mysql+redis

時間 2019-11-07

標籤 nginx+uwsgi+django+supervisor+mysql+redis nginx uwsgi django supervisor mysql redis 欄目 Nginx 简体版

原文原文鏈接

目錄css

1. 概述 3html

2. 安裝與配置 3node

2.1 django項目與應用建立 3python

2.2 uwsgi安裝與配置 6mysql

2.3 supervisor安裝與配置 8nginx

2.4 nginx安裝與做爲反向代理服務器的配置 10web

3. nginx+uwsgi優化 16sql

3.1 uwsgi進程線程數配置優化 16數據庫

3.2 Django數據庫鏈接參數CONN_MAX_AGE優化 20django

3.3 nginx中worker_rlimit_nofile與worker_connections優化 21

1. 概述

基於CentOS7.4的Nginx+UWsgi+Django+Supervisor是常見的web項目部署方式。爲方便項目部署，瞭解各個軟件配置，性能調優提供參考，以及高併發環境下一些異常修復與規避。

2. 安裝與配置

PtCloud可根據環境的不一樣進行靈活性的搭建和配置，但都須要使用PtCloud發行版光盤進行角色的選擇，並至少選擇一臺物理機做爲計算節點。根據根據安裝的不一樣狀況作如下的安裝說明。

2.1 Django項目與app建立

2.1.1 給python2.7安裝django1.10的庫

[root@bogon my_projects]# pip install django==1.10

2.1.2 建立django項目helloworld

[root@bogon my_projects]# django-admin.py startproject hellowold

2.1.3 建立第一個應用 application1

[root@bogon my_projects]# cd hellowold

[root@bogon hellowold]# django-admin.py startapp application1

2.1.4 修改setting.py中的ALLOWED_HOSTS

說明：ALLOWED_HOSTS後面所跟的屬性值是一個字符串列表值，這個字符串列表值表示當下這個Django站點能夠提供的host/domain(主機/域名)。*通配符去配置，另外當DEBUG設置爲False的時候必須配置這個配置。不然會拋出異常。

將/home/my_projects/helloworld/helloworld/setting.py中的ALLOWED_HOSTS修改成如下內容

ALLOWED_HOSTS = ['*']

2.1.5 運行django項目

[root@bogon hellowold]# ./manage.py runserver 0.0.0.0:1001

在另外一個終端執行如下命令，進行測試

[root@bogon my_projects]# curl http://127.0.0.1:1001

<!DOCTYPE html>

html * { padding:0; margin:0; }

body * { padding:10px 20px; }

body * * { padding:0; }

body { font:small sans-serif; }

body>div { border-bottom:1px solid #ddd; }

h1 { font-weight:normal; }

h2 { margin-bottom:.8em; }

h2 span { font-size:80%; color:#666; font-weight:normal; }

h3 { margin:1em 0 .5em 0; }

h4 { margin:0 0 .5em 0; font-weight: normal; }

table { border:1px solid #ccc; border-collapse: collapse; width:100%; background:white; }

tbody td, tbody th { vertical-align:top; padding:2px 3px; }

thead th {

padding:1px 6px 1px 3px; background:#fefefe; text-align:left;

font-weight:normal; font-size:11px; border:1px solid #ddd;

}

tbody th { width:12em; text-align:right; color:#666; padding-right:.5em; }

#summary { background: #e0ebff; }

#summary h2 { font-weight: normal; color: #666; }

#explanation { background:#eee; }

#instructions { background:#f6f6f6; }

#summary table { border:none; background:transparent; }

</style>

</head>

<body>

<h1>It worked!</h1>

<h2>Congratulations on your first Django-powered page.</h2>

</div>

<p>

Of course, you haven't actually done any work yet. Next, start your first app by running <code>python manage.py startapp [app_label]</code>.

</p>

</div>

<p>

You're seeing this message because you have <code>DEBUG = True</code> in your Django settings file and you haven't configured any URLs. Get to work!

</p>

</div>

</body></html>

2.2 uwsgi安裝與配置

2.2.1安裝uwsgi

[root@bogon hellowold]# yum -y install uwsgi

2.2.2 django項目hellowold中uwsgi.ini配置

[root@bogon hellowold]# pwd

/home/my_projects/hellowld

[root@bogon hellowold]# cat uwsgi.ini

[uwsgi]

chdir = /home/my_projects/helloworld

module = helloworld.wsgi:application

reload-mercy = 10

user = root

uid = root

master = True

harakiri-verbose = true

post-buffering = 65536

buffer-size = 65536

Harakiri = 30

threads = 30 # 與進程的乘積不大於數據庫容許的最大鏈接數

processes = 8 # 二倍於CPU核數

# socket = 127.0.0.1:7001 # nginx和 uwsgi間走的是 wsgi 協議，對應的nginx那邊也須要配置uwsgi的一些參數

http = 127.0.0.1:7001 # uwsgi將本端口的流按照http協議解析

chmod-socket = 664

vacuum = true

2.2.3 用uwsgi啓動django項目

若是步驟2.1.5啓動的程序還未中止，請用ctrl+c先中止，而後執行如下命令

[root@bogon hellowold]# uwsgi --ini /home/my_projects/helloworld/uwsgi.ini

在另外一個終端中查看結果，執行

[root@bogon hellowold]# curl http://127.0.0.1:1001 |grep Congratulations

<h2>The install worked successfully! Congratulations!</h2>

2.3 supervisor安裝與配置

2.3.1 supervisor安裝

[root@bogon hellowold]# yum install supervisor

2.3.2 supervisor配置

將/etc/supervisor/supervisor.conf修改成如下內容

[unix_http_server]

file=/var/run/supervisor/supervisor.sock ; (the path to the socket file)

[supervisord]

logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)

logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)

logfile_backups=10 ; (num of main logfile rotation backups;default 10)

loglevel=info ; (log level;default info; others: debug,warn,trace)

pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)

nodaemon=false ; (start in foreground if true;default false)

minfds=1024 ; (min. avail startup file descriptors;default 1024)

minprocs=200 ; (min. avail process descriptors;default 200)

[rpcinterface:supervisor]

supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]

serverurl=unix:///var/run/supervisor/supervisor.sock ; use a unix:// URL for a unix socket

[include]

files = /etc/supervisor/conf.d/*.conf

在/etc/supervisor/supervisor.d目錄下建立helloworld.conf內容以下

[program:helloworld]

directory = /home/my_projects/helloworld

command = uwsgi --ini /home/my_projects/helloworld/uwsgi.ini

user = root

autostart = true

autorestart = true

stopsignal = QUIT

redirect_stderr = true

loglevel = error

stdout_logfile = /var/log/uwsgi/client_uwsgi_out.log

stderr_logfile = /var/log/uwsgi/client_uwsgi_err.log

logfile_maxbytes = 10M

aogfile_maxbytes = 2M

若是步驟2.2.2啓動的程序還在運行，請先ctrl+c中止，而後執行如下命令

[root@bogon hellowold]# supervisord -c /etc/supervisor/supervisord.conf # 啓動supervisor服務

helloworld RUNNING pid 28899, uptime 0:00:03

其它supervisor相關命令

supervisorctl status # 查看supervisor管理的程序

supervisorctl stop all # 中止supervisor守護的全部進程

supervisorctl start all # 啓動supervisor守護的全部進程

supervisorctl stop helloworld # 中止supervisor守護的helloworld進程(也能夠是其它被守護的進程，該名稱在/etc/supervisor/conf.d/***.conf文件中[program:後配置)。

supervisorctl start helloworld # 啓動supervisor守護的helloworld進程

supervisorctl restart helloworld # 重啓supervisor守護的helloworld進程

2.4 Nginx安裝與做爲反向代理服務器的配置

2.4.1安裝nginx

[root@bogon my_projects]# yum -y install nginx

安裝好nginx後，nginx也爲本身建立好了用戶。在/etc/nginx/nginx.conf中user後面就是nginx執行時所使用的用戶。如：「user nginx;」

2.4.2配置nginx

除原有niginx.conf中的用戶不修改外，其他修改成如下內容。

# user nginx; # nginx 用戶爲原文件中的nginx用戶。

worker_processes auto;

pid /run/nginx.pid;

events {

worker_connections 768;

}

http {

sendfile on;

tcp_nopush on;

tcp_nodelay on;

keepalive_timeout 65;

types_hash_max_size 4096;

include /etc/nginx/mime.types;

default_type application/octet-stream;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE

ssl_prefer_server_ciphers on;

access_log /var/log/nginx/access.log;

error_log /var/log/nginx/error.log;

gzip on;

gzip_disable "msie6";

include /etc/nginx/conf.d/*.conf;

}

給helloworld項目設置反向代理規則

[root@bogon my_projects]# vim /etc/nginx/conf.d/helloworld.conf

server {

listen 80;

server_name 127.0.0.1; # ext-sandbox.51zhuan.com;

charset utf-8;

access_log /var/log/nginx/helloworld/nginx.access.log;

error_log /var/log/nginx/helloworld/nginx.error.log;

# Load configuration files for the default server block.

location / {

proxy_pass_header Server;

proxy_set_header Host $http_host;

proxy_redirect off;

proxy_set_header X-Real-IP $remote_addr;

proxy_pass http://127.0.0.1:1001;

}

error_page 404 /404.html;

location = /40x.html {

}

error_page 500 502 503 504 /50x.html;

location = /50x.html {

}

檢查配置文件nginx.conf的正確性：

[root@bogon my_projects]# nginx -t

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok

nginx: configuration file /etc/nginx/nginx.conf test is successful

2.1.3啓動 Nginx

Nginx 從新啓動命令以下：

[root@bogon my_projects]# service nginx restart

訪問站點

[root@bogon my_projects]# curl http://127.0.01

<!DOCTYPE html>

<html>

<head>

<title>Welcome to nginx!</title>

<style>

body {

width: 35em;

margin: 0 auto;

font-family: Tahoma, Verdana, Arial, sans-serif;

}

</style>

</head>

<body>

<h1>Welcome to nginALLOWED_HOSTS = ['*']

x!</h1>

<p>If you see this page, the nginx web server is successfully installed and

working. Further configuration is required.</p>

<p>For online documentation and support please refer to

<a href="http://nginx.org/">nginx.org</a>.<br/>

Commercial support is available at

<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>

</body>

</html>

1.2.4 Nginx 其餘命令

如下包含了 Nginx 經常使用的幾個命令：

nginx -s reload # 從新載入配置文件

nginx -s reopen # 重啓 Nginx

nginx -s stop # 中止 Nginx

service nginx status # 使用service查看nginx的狀態

service nginx start # 使用service啓動nginx

service nginx stop # 使用service中止nginx

service nginx restart # 使用service重啓nginx

3.nginx+uwsgi優化

3.1 http與socket路由方式

3.1.1 http路由方式

使用http路由方式，uwsgi與nginx之間也會以http協議進行通訊。在uwsgi內部會生成一個監聽已配置端口的服務器，轉發請求到由master進程管理的4個uWSGI worker組成的池中。

3.1.2 socket路由方式

nginx與uwsgi間使用wsgi協議進行socket通訊，uwsgi接受到nginx傳輸的數據後直接由workers進程對數據進行處理，而後返回給nginx，nginx再解析成http返回給客戶端。

修改uwsgi啓動方式爲socket，將uwsgi.ini中的http = 127.0.0.1:1001修改成socket = 127.0.0.1:1001

[root@bogon my_projects]# cd /home/my_projects/helloworld

[root@bogon helloworld]# cat uwsgi.ini

[uwsgi]

chdir = /home/agu/temp/helloworld

module = helloworld.wsgi:application

reload-mercy = 10

user = root

uid = root

master = True

harakiri-verbose = true

post-buffering = 65536

buffer-size = 65536

harakiri = 30

threads = 30

processes = 8

socket = 127.0.0.1:1001

# http = 127.0.0.1:1001

chmod-socket = 664

vacuum = true

重啓服務

[root@bogon helloworld]# supervisorctl restart helloworld

在使用socket方式時nginx須要與uwsgi適配，在/etc/nginx/nginx.conf文件http模塊中加入如下兩行

include uwsgi_params;

uwsgi_read_timeout 120;

建立/etc/nginx/uwsgi_params該文件內容以下

uwsgi_param QUERY_STRING $query_string;

uwsgi_param REQUEST_METHOD $request_method;

uwsgi_param CONTENT_TYPE $content_type;

uwsgi_param CONTENT_LENGTH $content_length;

uwsgi_param REQUEST_URI $request_uri;

uwsgi_param PATH_INFO $document_uri;

uwsgi_param DOCUMENT_ROOT $document_root;

uwsgi_param SERVER_PROTOCOL $server_protocol;

uwsgi_param REQUEST_SCHEME $scheme;

uwsgi_param HTTPS $https if_not_empty;

uwsgi_param REMOTE_ADDR $remote_addr;

uwsgi_param REMOTE_PORT $remote_port;

uwsgi_param SERVER_PORT $server_port;

uwsgi_param SERVER_NAME $server_name;

nginx重載配置文件或重啓nginx

[root@bogon helloworld]# nginx -s reload

在另外一個終端中查看結果，執行

[root@bogon hellowold]# curl http://127.0.0.1:1001 |grep successfully

<h2>The install worked successfully! Congratulations!</h2>

3.1 uwsgi進程線程數配置優化

3.1.1 uwsgi進程線程數配置

配置文件路徑/home/my_projects/helloworld/uwsgi.ini

修改processes進程數與threads進程數

3.1.2測試服務器配置與結果報表

3.1.2測試結論

當進程數爲4線程數爲60時平均吞吐量最高，錯誤率也相對較低。因此得出結論當進程數爲CPU核數2倍時，效率最高。

3.2 Django數據庫鏈接參數CONN_MAX_AGE優化

3.2.1 CONN_MAX_AGE參數配置

路徑/home/my_projects/helloworld/helloworld/settings.py database參數修改成如下內容

DATABASES = {

"default": {

"ENGINE": "django.db.backends.mysql",

"NAME": "integral_wall_sandbox",

"USER": "root",

"PASSWORD": "ztwl",

"HOST": "127.0.0.1",

"PORT": "3306",

"CONN_MAX_AGE": 28800, # 8 hours

}

測試報表

測試結論

1.當django數據庫鏈接中配置CONN_MAX_AGE參數時，只有views業務中從sql執行開始到本次請求結束的時間超過CONN_MAX_AGE時會當即斷開與數據庫的鏈接。若是執行時間少於CONN_MAX_AGE不會斷開鏈接。

2.當django數據庫鏈接中不配置CONN_MAX_AGE參數時，views業務中執行sql時會創建起與數據庫的鏈接，數據庫操做結束後當即斷開鏈接。

3.配置了CONN_MAX_AGE參數省去了頻繁鏈接與斷開數據的時間，因此服務器吞吐量略大於不配置CONN_MAX_AGE時的吞吐量。

4.uwsgi實際啓動進程數=1個主進程+uwsgi配置的進程數；主進程始終1個線程，子進程中的線程始終爲uwsgi中配置的線程數。

3.3 nginx中worker_rlimit_nofile與worker_connections優化

3.3.1 worker_rlimit_nofile解釋

本參數爲nginx工做進程改變打開最多文件描述符數目的限制。用來在不重啓主進程的狀況下增長限制，小於等於unix系統打開文件最大進程數的配置

3.3.2 當worker_rlimit_nofile不配置或者配置量太小，在高併發下可能出現的異常

異常日誌路徑/var/log/nginx/helloworld/nginx.access.log

異常日誌：socket() failed (24: Too many open files) while connecc ting to upstream

3.3.3 worker_connections含義

每個worker進程能併發處理（發起）的最大鏈接數。應小於等於worker_rlimit_nofile值。

3.3.4 當worker_connections配置量太小，在高併發下可能出現的異常

異常日誌路徑/var/log/nginx/helloworld/nginx.access.log

異常日誌：socket() failed (24: Too many open files) while connecc ting to upstream

更多相關文章...

相關標籤/搜索

每日一句

每一个你不满意的现在，都有一个你没有努力的曾经。