Nginx: PV、UV、獨立IP
作網站的都知道，日常常常要查詢下網站PV、UV等網站的訪問數據，固然若是網站作了CDN的話，nginx本地的日誌就沒什麼意義了，下面就對nginx網站的日誌訪問數據作下統計；html
概念：node
- UV(Unique Visitor)：獨立訪客，將每一個獨立上網電腦（以cookie爲依據）視爲一位訪客，一天以內（00:00-24:00），訪問您網站的訪客數量。一天以內相同cookie的訪問只被計算1次
- PV（Page View）：訪問量,即頁面瀏覽量或者點擊量,用戶每次對網站的訪問均被記錄1次。用戶對同一頁面的屢次訪問，訪問量值累計
- 統計獨立IP：00:00-24:00內相同IP地址只被計算一次,作網站優化的朋友最關心這個
先聲明下環境，這次運行的nginx版本1.7，後端Tomcat運行的是動態交互程序（需進行用戶認證，若是是靜態頁面則抓不到cache值，$http_cookie是空值），就是這樣；python
nginx日誌文件配置nginx
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
|
http {
include mime.types;
default_type application
/
octet
-
stream;
log_format main
'$remote_addr - [$time_local] "$request" '
' - $status "User_Cookie:$guid" '
;
#User_Cookie爲日誌顯示字符，$guid爲變量，具體內容在下面定義，也可在日誌格式裏寫入$http_cookie 顯示完整的cookie內容<br>
sendfile on;
keepalive_timeout
65
;
upstream backserver {
ip_hash;
server
1.1
.
2.2
:
8080
;
server
1.1
.
2.3
:
8080
;
}
server {
listen
80
;
server_name localhost;
#if ( $http_cookie ~* "(.*)$") 匹配全部內容
if
( $http_cookie ~
*
"CSID=([A-Z0-9]*)"
){
set
$guid $
1
;
}
#只匹配CSID字符信息，此處爲正則表達式<br>
access_log logs
/
host.access.log main;
location ~
*
^(.
*
)$ {
#limit_req zone=allips burst=1 nodelay;
proxy_pass http:
/
/
backserver;
proxy_set_header Host $host;
proxy_set_header X
-
Real
-
IP $remote_addr;
proxy_set_header REMOTE
-
HOST $remote_addr;
proxy_set_header X
-
Forwarded
-
For $proxy_add_x_forwarded_for;
client_max_body_size
8m
;
}
error_page
500
502
503
504
/
50x
.html;
location
=
/
50x
.html {
root html;
}
}
|
注：$http_cookie這個裏面的值是一個一個cookie的值，中間以「;」分隔web
日誌輸出格式正則表達式
|
1
2
3
4
5
6
7
8
9
|
192.168
.
40.2
-
[
02
/
Nov
/
2016
:
15
:
44
:
35
+
0800
]
"GET /wcm/app/main/refresh.jsp?r=1478072325778 HTTP/1.1"
-
200
"User_Cookie:7F00000122A5597C46607B1C0A7EC016"
192.168
.
40.2
-
[
02
/
Nov
/
2016
:
15
:
44
:
35
+
0800
]
"GET /webpic/W0201611/W020161102/W020161102566715167404.jpg HTTP/1.1"
-
200
"User_Cookie:7F00000122A5597C46607B1C0A7EC016"
119.255
.
31.109
-
[
02
/
Nov
/
2016
:
15
:
44
:
36
+
0800
]
"GET /wcm/app/main/refresh.jsp?r=1478072510132 HTTP/1.1"
-
200
"User_Cookie:7F000001237921BE9237838AEC65704D"
119.255
.
31.109
-
[
02
/
Nov
/
2016
:
15
:
44
:
36
+
0800
]
"GET /wcm/app/message/message_query_service.jsp?READFLAG=0&MSGTYPES=1%2C2%2C3 HTTP/1.1"
-
200
"User_Cookie:7F000001237921BE9237838AEC65704D"
192.168
.
40.2
-
[
02
/
Nov
/
2016
:
15
:
44
:
37
+
0800
]
"GET /wcm/app/message/message_query_service.jsp?READFLAG=0&MSGTYPES=1%2C2%2C3 HTTP/1.1"
-
200
"User_Cookie:7F00000123D3BF2345115EAAC21F71E0"
192.168
.
40.2
-
[
02
/
Nov
/
2016
:
15
:
44
:
37
+
0800
]
"GET /wcm/app/message/message_query_service.jsp?READFLAG=0&MSGTYPES=1%2C2%2C3 HTTP/1.1"
-
200
"User_Cookie:7F00000123EF73896DF98EDA9950944E"
192.168
.
40.2
-
[
02
/
Nov
/
2016
:
15
:
44
:
37
+
0800
]
"GET /wcm/app/message/message_query_service.jsp?READFLAG=0&MSGTYPES=1%2C2%2C3 HTTP/1.1"
-
200
"User_Cookie:7F00000123FE0F9C397E1A8F0C4F044B"
192.168
.
40.2
-
[
02
/
Nov
/
2016
:
15
:
44
:
37
+
0800
]
"GET /wcm/app/main/refresh.jsp?r=1478072511427 HTTP/1.1"
-
200
"User_Cookie:7F00000123A465B7EA1DE0AF0AE671B7"
119.255
.
31.109
-
[
02
/
Nov
/
2016
:
15
:
44
:
38
+
0800
]
"GET /wcm/app/message/message_query_service.jsp?READFLAG=0&MSGTYPES=1%2C2%2C3 HTTP/1.1"
-
200
"User_Cookie:7F00000123D89B11302DF80AE773C900"
|
PV統計後端
可統計單個連接地址訪問量：cookie
|
1
|
[root@localhost logs]
# grep index.shtml host.access.log | wc -l
|
總PV量：app
|
1
|
[root@localhost logs]
# awk '{print $6}' host.access.log | wc -l
|
獨立IP jsp
|
1
|
[root@localhost logs]
# awk '{print $1}' host.access.log | sort -r |uniq -c | wc -l
|
UV統計
|
1
|
[root@localhost logs]
# awk '{print $10}' host.access.log | sort -r |uniq -c |wc -l
|
Cookie 測試頁面
關於種cookie，能夠使用下面的html代碼，編輯，添加須要種的cookie
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
|
#index.html
<!DOCTYPE html PUBLIC
"-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"
>
<html xmlns
=
"http://www.w3.org/1999/xhtml"
>
<head>
<meta http
-
equiv
=
"Content-Type"
content
=
"text/html; charset=gbk"
>
<meta http
-
equiv
=
"Refresh"
content
=
"10"
>
/
/
爲了方便測試，每
10
秒刷新一次頁面
<
/
head>
<body>
<h1>test.test.com域測試<
/
h1>
下面列出了該域的cookie<br>
<p>
<script>
document.cookie
=
"guid=A1UD8E5512451111111111"
;
/
/
種cookie，追加
document.cookie
=
"city=beijing"
;
/
/
種cookie，追加
document.write(document.cookie);
/
/
列出已經存在的
<
/
script>
<
/
p>
<
/
body>
<
/
html>
|