nginx限制請求之三：Nginx+Lua+Redis 對請求進行限制 nginx限制請求之一：（ngx_http_limit_conn_module）模塊 分佈式限流之一：redis+lua 實現分

相關文章：

《高可用服務設計之二：Rate limiting 限流與降級》

《nginx限制請求之一：（ngx_http_limit_conn_module）模塊》

《nginx限制請求之二：（ngx_http_limit_req_module）模塊》

《nginx限制請求之三：Nginx+Lua+Redis 對請求進行限制》

《nginx限制請求之四：目錄進行IP限制》

《分佈式限流之一：redis+lua 實現分佈式令牌桶，高併發限流》

 

1、概述

需求：全部訪問/myapi/**的請求必須是POST請求，並且根據請求參數過濾不符合規則的非法請求(黑名單), 這些請求一概不轉發到後端服務器(Tomcat)

實現思路：經過在Nginx上進行訪問限制，經過Lua來靈活實現業務需求，而Redis用於存儲黑名單列表。

相關nginx上lua或redis的使用方式能夠參考我以前寫的一篇文章:

=======================================================

1、概述：

1.研究目標：nginx中使用lua腳本，及nginx直接訪問mysql,redis

2.須要安裝的內容：

openresty,mysql,redis

3.OpenResty （也稱爲 ngx_openresty）是一個全功能的 Web 應用服務器。它打包了標準的 Nginx 核心，不少的經常使用的第三方模塊，以及它們的大多數依賴項。http://openresty.org/cn/index.html

 

 

2、安裝說明

 

0.環境準備

$yum install -y gcc gcc-c++ readline-devel pcre-devel openssl-devel tcl perl

一、安裝drizzle http://wiki.nginx.org/HttpDrizzleModule

cd /usr/local/src/ 
wget http://openresty.org/download/drizzle7-2011.07.21.tar.gz 
tar xzvf drizzle-2011.07.21.tar.gz 
cd drizzle-2011.07.21/ 
./configure --without-server 
make libdrizzle-1.0 
make install-libdrizzle-1.0 
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

二、安裝openresty 
wget http://openresty.org/download/ngx_openresty-1.7.2.1.tar.gz 
tar xzvf ngx_openresty-1.7.2.1.tar.gz 
cd ngx_openresty-1.7.2.1/ 
./configure --with-http_drizzle_module 
gmake 
gmake install

 

3、nginx配置nginx.conf

 

/usr/local/openresty/nginx/conf/nginx.conf

# 添加MySQL配置(drizzle)  upstream backend {      drizzle_server 127.0.0.1:3306 dbname=test user=root password=123456 protocol=mysql;      drizzle_keepalive max=200 overflow=ignore mode=single;  } server {      listen       80;      server_name  localhost;     #charset koi8-r;      #access_log  logs/host.access.log  main;     location / {          root   html;          index  index.html index.htm;      }     location /lua {          default_type text/plain;          content_by_lua 'ngx.say("hello, lua")';      }     location /lua_redis {          default_type text/plain;          content_by_lua_file /usr/local/lua_test/redis_test.lua;      }      location /lua_mysql {              default_type text/plain;              content_by_lua_file /usr/local/lua_test/mysql_test.lua;      }     location @cats-by-name {          set_unescape_uri $name $arg_name;          set_quote_sql_str $name;          drizzle_query 'select * from cats where name=$name';          drizzle_pass backend;          rds_json on;      }     location @cats-by-id {          set_quote_sql_str $id $arg_id;          drizzle_query 'select * from cats where id=$id';          drizzle_pass backend;          rds_json on;      }     location = /cats {          access_by_lua '              if ngx.var.arg_name then                  return ngx.exec("@cats-by-name")              end             if ngx.var.arg_id then                  return ngx.exec("@cats-by-id")              end          ';         rds_json_ret 400 "expecting \"name\" or \"id\" query arguments";      }     # 經過url匹配出name，並編碼防止注入，最後以json格式輸出結果      location ~ '^/mysql/(.*)' {          set $name $1;          set_quote_sql_str $quote_name $name;          set $sql "SELECT * FROM cats WHERE name=$quote_name";          drizzle_query $sql;          drizzle_pass backend;          rds_json on;      }     # 查看MySQL服務狀態      location /mysql-status {          drizzle_status;      }  }

 

4、lua測試腳本

/usr/local/lua_test/redis_test.lua

local redis = require "resty.redis" local cache = redis.new() cache.connect(cache, '127.0.0.1', '6379') local res = cache:get("foo") if res==ngx.null then ngx.say("This is Null") return end ngx.say(res)

 

/usr/local/lua_test/mysql_test.lua

local mysql = require "resty.mysql" local db, err = mysql:new() if not db then ngx.say("failed to instantiate mysql: ", err) return end db:set_timeout(1000) -- 1 sec -- or connect to a unix domain socket file listened -- by a mysql server: -- local ok, err, errno, sqlstate = -- db:connect{ -- path = "/path/to/mysql.sock", -- database = "ngx_test", -- user = "ngx_test", -- password = "ngx_test" } local ok, err, errno, sqlstate = db:connect{ host = "127.0.0.1", port = 3306, database = "test", user = "root", password = "123456", max_packet_size = 1024 * 1024 } if not ok then ngx.say("failed to connect: ", err, ": ", errno, " ", sqlstate) return end ngx.say("connected to mysql.") local res, err, errno, sqlstate = db:query("drop table if exists cats") if not res then ngx.say("bad result: ", err, ": ", errno, ": ", sqlstate, ".") return end res, err, errno, sqlstate = db:query("create table cats " .. "(id serial primary key, " .. "name varchar(5))") if not res then ngx.say("bad result: ", err, ": ", errno, ": ", sqlstate, ".") return end ngx.say("table cats created.") res, err, errno, sqlstate = db:query("insert into cats (name) " .. "values (\'Bob\'),(\'\'),(null)") if not res then ngx.say("bad result: ", err, ": ", errno, ": ", sqlstate, ".") return end ngx.say(res.affected_rows, " rows inserted into table cats ", "(last insert id: ", res.insert_id, ")") -- run a select query, expected about 10 rows in -- the result set: res, err, errno, sqlstate = db:query("select * from cats order by id asc", 10) if not res then ngx.say("bad result: ", err, ": ", errno, ": ", sqlstate, ".") return end local cjson = require "cjson" ngx.say("result: ", cjson.encode(res)) -- put it into the connection pool of size 100, -- with 10 seconds max idle timeout local ok, err = db:set_keepalive(10000, 100) if not ok then ngx.say("failed to set keepalive: ", err) return end -- or just close the connection right away: -- local ok, err = db:close() -- if not ok then -- ngx.say("failed to close: ", err) -- return -- end ';

 

5、驗證結果

curl測試

$ curl 'http://127.0.0.1/lua_test'  hello, lua $ redis-cli set foo 'hello,lua-redis'  OK  $ curl 'http://127.0.0.1/lua_redis'  hello,lua-redis $ curl 'http://127.0.0.1/lua_mysql'  connected to mysql.  table cats created.  3 rows inserted into table cats (last insert id: 1)  result: [{"name":"Bob","id":"1"},{"name":"","id":"2"},{"name":null,"id":"3"}] $ curl 'http://127.0.0.1/cats'  {"errcode":400,"errstr":"expecting \"name\" or \"id\" query arguments"} $ curl 'http://127.0.0.1/cats?name=bob'  [{"id":1,"name":"Bob"}] $ curl 'http://127.0.0.1/cats?id=2'  [{"id":2,"name":""}] $ curl 'http://127.0.0.1/mysql/bob'  [{"id":1,"name":"Bob"}] $ curl 'http://127.0.0.1/mysql-status'  worker process: 32261 upstream backend    active connections: 0    connection pool capacity: 0    servers: 1    peers: 1

 

 

6、參考資料

1.openresty http://openresty.org/cn/index.html

2.tengine  http://tengine.taobao.org/documentation_cn.html

 

如何安裝nginx_lua_module模塊 
http://www.cnblogs.com/yjf512/archive/2012/03/27/2419577.html

nginx+lua 項目使用記（二） 
http://blog.chinaunix.net/uid-26443921-id-3213879.html

nginx_lua模塊基於mysql數據庫動態修改網頁內容 
https://www.centos.bz/2012/09/nginx-lua-mysql-dynamic-modify-content/

突破log_by_lua中限制Cosocket API的使用 
http://17173ops.com/2013/11/11/resolve-cosocket-api-limiting-in-log-by-lua.shtml

17173 Ngx_Lua使用分享 
http://17173ops.com/2013/11/01/17173-ngx-lua-manual.shtml

關於 OPENRESTY 的兩三事 
http://zivn.me/?p=157

Nginx_Lua 
http://www.ttlsa.com/nginx/nginx-lua/

Nginx 第三方模塊-漫談緣起 
http://www.cnblogs.com/yjf512/archive/2012/03/30/2424726.html

CentOS6.4 安裝OpenResty和Redis 並在Nginx中利用lua簡單讀取Redis數據 
http://www.cnblogs.com/kgdxpr/p/3550633.html

Nginx與Lua 
http://huoding.com/2012/08/31/156

由Lua 粘合的Nginx生態環境 
http://blog.zoomquiet.org/pyblosxom/oss/openresty-intro-2012-03-06-01-13.html

Nginx 第三方模塊試用記 
http://chenxiaoyu.org/2011/10/30/nginx-modules.html

 

agentzh 的 Nginx 教程（版本 2013.07.08） 
http://openresty.org/download/agentzh-nginx-tutorials-zhcn.html

 

CentOS下Redis 2.2.14安裝配置詳解 
http://www.cnblogs.com/hb_cattle/archive/2011/10/22/2220907.html

 

nginx安裝 
http://blog.csdn.net/gaojinshan/article/details/37603157

=======================================================

 

2、具體實現

1.lua代碼

本例中限制規則包括(post請求，ip地址黑名單，請求參數中imsi,tel值和黑名單)

 1 -- access_by_lua_file '/usr/local/lua_test/my_access_limit.lua';
 2 ngx.req.read_body()
 3 
 4 local redis = require "resty.redis"
 5 local red = redis.new()
 6 red.connect(red, '127.0.0.1', '6379')
 7 
 8 local myIP = ngx.req.get_headers()["X-Real-IP"]
 9 if myIP == nil then
10    myIP = ngx.req.get_headers()["x_forwarded_for"]
11 end
12 if myIP == nil then
13    myIP = ngx.var.remote_addr
14 end
15         
16 if ngx.re.match(ngx.var.uri,"^(/myapi/).*$") then
17     local method = ngx.var.request_method
18     if method == 'POST' then
19         local args = ngx.req.get_post_args()
20         
21         local hasIP = red:sismember('black.ip',myIP)
22         local hasIMSI = red:sismember('black.imsi',args.imsi)
23         local hasTEL = red:sismember('black.tel',args.tel)
24         if hasIP==1 or hasIMSI==1 or hasTEL==1 then
25             --ngx.say("This is 'Black List' request")
26             ngx.exit(ngx.HTTP_FORBIDDEN)
27         end
28     else
29         --ngx.say("This is 'GET' request")
30         ngx.exit(ngx.HTTP_FORBIDDEN)
31     end
32 end

 

2.nginx.conf

location / {
root html;
index index.html index.htm;

access_by_lua_file /usr/local/lua_test/my_access_limit.lua;

proxy_pass http://127.0.0.1:8080;
client_max_body_size 1m;
}

 

3.添加黑名單規則數據

#redis-cli sadd black.ip '153.34.118.50'
#redis-cli sadd black.imsi '460123456789' 
#redis-cli sadd black.tel '15888888888'

 

能夠經過redis-cli smembers black.imsi 查看列代表細

 

4.驗證結果

#curl -d "imsi=460123456789&tel=15800000000" "http://www.mysite.com/myapi/abc"