基於Varnish 3.0.4的網站靜態加速

Varnish 的一些特色:
php

(1)、是基於內存緩存,重啓後數據將消失;css

(2)、利用虛擬內存方式,I/O 性能好;html

(3)、支持設置 0~60 秒內的精確緩存時間;web

(4)、VCL 配置管理比較靈活;正則表達式

(5)、32位機器上緩存文件大小爲最大2G;算法

(6)、具備強大的管理功能,例如 top、stat、admin、list 等;shell

(7)、狀態機設計巧妙,結構清晰;vim

(8)、利用二叉堆管理緩存文件,達到積極刪除目的。 後端

-----------------------------------------------------------------------------------------------------------------------------------------------緩存

Varnish 的 Storage 方式可分爲兩種:

1)、Malloc 經過 malloc 獲取內存;

2)、Mmap file 建立大文件,經過二分法分段映射成 1G 之內的大塊。


以 Mmap file 的緩存方式啓動 I/O 也會造成瓶頸,緣由主要是 Varnish 緩存的數據先會刷到磁盤上,而後在一次性讀到內存中,這在訪問量大的時候同時也會對 I/O 形成很大的壓力。Malloc 緩存方式雖然對 I/O 沒有壓力,因全部緩存數據都寫到內存中。


Malloc 方式啓動:

/usr/local/varnish/sbin/varnishd -u nobody -g nogroup -f /usr/local/varnish/etc/varnish.vcl -s malloc,4G -w 50,51200,120 -a 192.168.10:80 -T 127.0.0.1:8080


Mmap file 方式啓動:

/usr/local/varnish/sbin/varnishd -u nobody -g nogroup -f /usr/local/varnish/etc/varnish.vcl -s file,/data/varnish/varnish_storage.bin,4G -w 50,51200,120 -a 192.168.10:80 -T 127.0.0.1:8080

-----------------------------------------------------------------------------------------------------------------------------------------------

Varnish 進程的工做模式:

Varnish 啓動或有2個進程 master(management) 進程和 child(worker) 進程。master 讀入存儲配置命令,進行初始化,而後 fork,監控 child。child 則分配線程進行 cache 工做,child 還會作管理線程和生成不少 worker 線程。


child 進程主線程初始化過程當中,將存儲大文件整個加載到內存中,若是該文件超出系統的虛擬內存,則會減小原來配置 mmap 大小,而後繼續加載,這時候建立並初始化空閒存儲結構體,放在存儲管理的 struct 中,等待分配。


接着 Varnish 某個負責接受新 http 鏈接的線程開始等待用戶,若是有新的 http 鏈接,可是這個線程只負責接收,而後喚醒等待線程池中的 work 線程,進行請求處理。


worker 線程讀入 uri 後,將會查找已有的 object,命中直接返回,沒有命中,則會從後端服務器中取出來,放到緩存中。若是緩存已滿,會根據 LRU 算法,釋放舊的 object。對於釋放緩存,有一個超時線程會檢測緩存中全部 object 的生命週期,若是緩存過時 (ttl),則刪除,釋放相應的存儲內存。

-----------------------------------------------------------------------------------------------------------------------------------------------

Varnish 的優勢:

(1)、Varnish 的穩定性很高,二者在完成相同負荷的工做時,Squid 服務器發生故障的概率要高於 Varnish,由於使用Squid 要常常重啓;

(2)、Varnish 訪問速度更快,Varnish 採用了Visual Page Cache技術,全部緩存數據都直接從內存讀取,而 Squid 是從硬盤讀取,於是 Varnish 在訪問速度方面會更快;

(3)、Varnish 能夠支持更多的併發鏈接,由於 Varnish 的 TCP 鏈接釋放要比 Squid 快。於是在高併發鏈接狀況下能夠支持更多 TCP 鏈接;

(4)、Varnish 能夠經過管理端口,使用正則表達式批量的清除部分緩存,而 Squid 是作不到的;

(5)、Squid 屬因而單進程使用單核 CPU,但 Varnish 是經過 fork 形式打開多進程來作處理,因此是合理的使用全部核來處理相應的請求。


與傳統的 Squid 相比,Varnish 也有缺點:

1)、Varnish 進程一旦 Hang、Crash 或者重啓,緩存數據都會從內存中徹底釋放,此時全部請求都會發送到後端服務器,在高併發狀況下,會給後端服務器形成很大壓力;

2)、在 Varnish 使用中,若是單個 url 的請求經過 HA/F5(負載均衡)、每次請求不一樣的 Varnish 服務器中,被請求Varnish 服務器都會被穿透到後端,並且一樣的請求會在多臺服務器上緩存,也會形成 Varnish 的緩存的資源浪費,也會形成性能降低。


解決方案:

1)、綜上所述在訪問量很大的狀況下推薦使用 Varnish 的內存緩存方式啓動,並且後面須要跟多臺 Squid 服務器。主要爲了防止前面的 Varnish 服務被重啓的狀況下,前期確定會有不少的穿透,這樣 Squid 能夠擔當第二層 CACHE,並且也彌補了 Varnish 緩存在內存中重啓都會釋放的問題;

2)、這樣的問題能夠在負載均衡上作 url 哈希,讓單個 url 請求固定請求到一臺 Varnish 服務器上,能夠解決該問題。


注:以上內容是在借鑑別人總結的基礎上,作了一些精簡、修正,並加入了一些本身的總結。


-----------------------------------------------------------------------------------------------------------------------------------------------

系統環境:SUSE Linux Enterprise Server 10 SP1 (x86_64)

硬件配置:8G內存、8核CPU


一、編譯安裝

ftp://invisible-island.net/ncurses/ncurses-5.9.tar.gz

# tar xvzf ncurses-5.9.tar.gz

# ./configure --prefix=/usr/local

# make LAGS=-fPIC

# make LAGS=-fPIC install


http://downloads.sourceforge.net/project/pcre/pcre/8.32/pcre-8.32.tar.gz

# tar xvzf  pcre-8.32.tar.gz

# cd pcre-8.32

# ./configure --prefix=/usr/local

# make

# make install


ftp://ftp.gnu.org/gnu/readline/readline-6.2.tar.gz

# tar xvzf readline-6.2.tar.gz

# ./configure --prefix=/usr/local

# make

# make install


http://repo.varnish-cache.org/source/varnish-3.0.4.tar.gz

# tar xvzf varnish-3.0.4.tar.gz

# ./configure --prefix=/usr/local/varnish

# make

# make install

-----------------------------------------------------------------------------------------------------------------------------------------------

二、後端WEB服務健康檢測

# vim /usr/local/varnish/etc/health_check.vcl

probe backend_healthcheck {
    .interval = 5s;
    .timeout = 3s;
    .window = 10;
    .threshold = 8;
         
    .request =
    "GET /index.html HTTP/1.1"
    "Host: mycheckweb.mytest.com"
    "Connection: close"
    "Accept-Encoding: foo/bar";
}

-----------------------------------------------------------------------------------------------------------------------------------------------

三、後端WEB服務定義

# vim /usr/local/varnish/etc/hosts/10.160.22.88_8080.conf

backend WEBSRV_10_160_22_88_8080 {
    .host = "10.160.22.88";
    .port = "8080";
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
    .connect_timeout = 50s;
    .between_bytes_timeout = 30s;
    .first_byte_timeout = 30s;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
    .probe = backend_healthcheck;
}

# vim /usr/local/varnish/etc/hosts/10.173.146.35_8080.conf

backend WEBSRV_10_173_146_35_8080 {
    .host = "10.173.146.35";
    .port = "8080";
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 
    .connect_timeout = 50s;
    .between_bytes_timeout = 30s;
    .first_byte_timeout = 30s;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 
    .probe = backend_healthcheck;
}

-----------------------------------------------------------------------------------------------------------------------------------------------

四、集羣定義

# vim /usr/local/varnish/etc/cluster.vcl

include "/usr/local/varnish/etc/health_check.vcl";
include "/usr/local/varnish/etc/hosts/10.160.22.88_8080.conf";
include "/usr/local/varnish/etc/hosts/10.173.146.35_8080.conf";
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         
director CLUSTER_BACKEND_SERVER round-robin {
    { .backend = WEBSRV_10_160_22_88_8080; }
    { .backend = WEBSRV_10_173_146_35_8080; }
}

-----------------------------------------------------------------------------------------------------------------------------------------------

五、Varnish主配置文件

# vim /usr/local/varnish/etc/varnish.vcl

include "/usr/local/varnish/etc/cluster.vcl";
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
acl allow_purge_cache {
    "127.0.0.1";
    "10.0.0.0"/8;
    "172.0.0.0"/8;
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_recv {
    if (req.request == "PURGE") {
        if (!client.ip ~ allow_purge_cache) {
            error 405 "Not Allowed.";
        }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
        return (lookup);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.http.host ~ "^(.*).mytest.com") {
        set req.backend = CLUSTER_BACKEND_SERVER;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 動態資源直接拋到後端服務器
    if (req.url ~ "\.(php|asp|aspx|jsp|do|ashx|shtml)($|\?)") {
        return (pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 靜態資源須要去除cookie信息
    if (req.request == "GET" && req.url ~ "\.(css|js|bmp|png|gif|jpg|jpeg|ico|gz|tgz|bz2|tbz|zip|rar|mp3|mp4|ogg|swf|flv)($|\?)") {
        unset req.http.cookie;
        return (lookup);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.restarts == 0) {
        if (req.http.x-forwarded-for) {
            set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.http.Cache-Control ~ "no-cache") {
        return (pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.request != "GET" &&
        req.request != "HEAD" &&
        req.request != "PUT" &&
        req.request != "POST" &&
        req.request != "TRACE" &&
        req.request != "OPTIONS" &&
        req.request != "DELETE") {
        return (pipe);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.request != "GET" && req.request != "HEAD") {
        return (pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.http.Authorization || req.http.Cookie) {
        return (pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 靜態資源壓縮
    if (req.http.Accept-Encoding) {
        if (req.url ~ "\.(bmp|png|gif|jpg|jpeg|ico|gz|tgz|bz2|tbz|zip|rar|mp3|mp4|ogg|swf|flv)$") {
            remove req.http.Accept-Encoding;
        } elseif (req.http.Accept-Encoding ~ "gzip") {
            set req.http.Accept-Encoding = "gzip";
        } elseif (req.http.Accept-Encoding ~ "deflate") {
            set req.http.Accept-Encoding = "deflate";
        } else {
            remove req.http.Accept-Encoding;
        }
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 防盜鏈設置
    if (req.http.referer ~ "http://.*") {
        if (!(req.http.referer ~ "http://.*\.qq\.com" ||
            req.http.referer ~ "http://.*\.baidu\.com" ||
            req.http.referer ~ "http://.*\.google\.com.*" ||
            req.http.referer ~ "http://.*\.sogou\.com" ||
            req.http.referer ~ "http://.*\.soso\.com" ||
            req.http.referer ~ "http://.*\.so\.com")) {
            set req.http.host = "www.mytest.com";
            set req.url = "/";
        }
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (!req.backend.healthy) {
        unset req.http.Cookie;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 跳過緩存大文件
    if (req.http.x-pipe && req.restarts > 0) {
        unset req.http.x-pipe;
        return (pipe);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 若backend是健康的,則僅grace 5s,若是backend不健康,則grace 1m,主要用於提升併發時的吞吐率
    if (req.backend.healthy) {
        set req.grace = 5s;
    } else {
        set req.grace = 1m;
    }
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_pipe {
    return (pipe);
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_pass {
    if (req.request == "PURGE") {
        error 502 "PURGE on a passed object";
    }
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_hash {
    hash_data(req.url);
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.http.host) {
        hash_data(req.http.host);
    } else {
        hash_data(server.ip);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.http.Accept-Encoding ~ "gzip") {
        hash_data("gzip");
    } elseif (req.http.Accept-Encoding ~ "deflate") {
        hash_data("deflate");
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    return (hash);
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_hit {
    if (req.request == "PURGE") {
        purge;
        error 200 "Purged.";
    }
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_miss {
    if (req.request == "PURGE") {
        purge;
        error 404 "Purged.";
    }
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_fetch {
    ## 確保全部Cache中的內容在TTL過時後5分鐘內不被刪除,以應對高併發的場合
    set beresp.grace = 5m;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (beresp.http.Set-Cookie) {
        return (hit_for_pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 若是返回頭有Cache-Control,則刪除Set-Cookie頭
    if (beresp.http.Cache-Control && beresp.ttl > 0s) {
        set beresp.grace = 1m;
        unset beresp.http.Set-Cookie;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 不緩存大於10MB的資源文件
    if (beresp.http.Content-Length ~ "[0-9]{8,}") {
        set req.http.x-pipe = "1";
        return (restart);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.url ~ "\.(php|asp|aspx|jsp|do|ashx|shtml)($|\?)") {
        return (hit_for_pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (req.request == "GET" && req.url ~ "\.(css|js|bmp|png|gif|jpg|jpeg|ico|gz|tgz|bz2|tbz|zip|rar|mp3|mp4|ogg|swf|flv)($|\?)") {
        unset beresp.http.set-cookie;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 若是返回頭沒有Cache-Control,則標記爲hit_for_pass,強制後續請求回源
    if ((!beresp.http.Cache-Control && !beresp.http.Expires) ||
         beresp.http.Pragma ~ "no-cache" ||
         beresp.http.Cache-Control ~ "(no-cache|no-store|private)") {
        set beresp.ttl = 120s;
        return (hit_for_pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Vary == "*") {
        set beresp.ttl = 120s;
        return (hit_for_pass);
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 對不一樣類型靜態資源進行緩存時間設置
    if (req.request == "GET" && req.url ~ "\.(css|js|bmp|png|gif|jpg|jpeg|ico)($|\?)") {
        set beresp.ttl = 15m;
    } elseif (req.request == "GET" && req.url ~ "\.(gz|tgz|bz2|tbz|zip|rar|mp3|mp4|ogg|swf|flv)($|\?)") {
        set beresp.ttl = 30m;
    } else {
        set beresp.ttl = 10m;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    return (deliver);
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_deliver {
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT from " + req.http.host;
        set resp.http.X-Cache-Hits = obj.hits;
    } else {
        set resp.http.X-Cache = "MISS from " + req.http.host;
    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    ## 去掉沒必要要的頭信息
    unset resp.http.X-Powered-By;
    unset resp.http.Server;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    unset resp.http.Via;
    unset resp.http.X-Varnish;
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
    unset resp.http.Age;
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_error {
    if (obj.status == 503 && req.restarts < 5) {
        set obj.http.X-Restarts = req.restarts;
        return (restart);
    }
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_init {
    return (ok);
}
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
sub vcl_fini {
    return (ok);
}

-----------------------------------------------------------------------------------------------------------------------------------------------

六、Varnish啓動參數配置文件

# vim /usr/local/varnish/etc/varnish.conf

# Configuration file for varnish
#
# /etc/init.d/varnishd expects the variable $DAEMON_OPTS to be set from this
# shell script fragment.
#
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
# Maximum number of open files (for ulimit -n)
NFILES=131072
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
# Locked shared memory (for ulimit -l)
# Default log size is 82MB + header
MEMLOCK=82000
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
# Maximum number of threads (for ulimit -u)
NPROCS="unlimited"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
# Set this to 1 to make init script reload try to switch vcl without restart.
# To make this work, you need to set the following variables
# explicit: VARNISH_VCL_CONF, VARNISH_ADMIN_LISTEN_ADDRESS,
# VARNISH_ADMIN_LISTEN_PORT, VARNISH_SECRET_FILE, or in short,
# use Alternative 3, Advanced configuration, below
RELOAD_VCL=1
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Main configuration file. You probably want to change it :)
VARNISH_VCL_CONF=/usr/local/varnish/etc/varnish.vcl
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Default address and port to bind to
# # Blank address means all IPv4 and IPv6 interfaces, otherwise specify
# # a host name, an IPv4 dotted quad, or an IPv6 address in brackets.
VARNISH_LISTEN_ADDRESS=0.0.0.0
VARNISH_LISTEN_PORT=80
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Telnet admin interface listen address and port
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=8080
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Shared secret file for admin interface
#VARNISH_SECRET_FILE=/etc/varnish/secret
#
# # The minimum number of worker threads to start
VARNISH_MIN_THREADS=10
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # The Maximum number of worker threads to start
VARNISH_MAX_THREADS=5000
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Idle timeout for worker threads
VARNISH_THREAD_TIMEOUT=120
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Cache file location
VARNISH_STORAGE_FILE=/data/varnish/varnish_storage.bin
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Cache file size: in bytes, optionally using k / M / G / T suffix,
# # or in percentage of available disk space using the % suffix.
VARNISH_STORAGE_SIZE=3G
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Backend storage specification
#VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"
VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Default TTL used when the backend does not specify one
VARNISH_TTL=120
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # Other optimization parameter
HTTP_RESP_HDR_LEN="http_resp_hdr_len=8192"
HTTP_MAX_HDR="http_max_hdr=256"
HTTP_REQ_HDR_LEN="http_req_hdr_len=8192"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
THREAD_POOLS="thread_pools=8"
THREAD_POOL_MIN="thread_pool_min=50"
THREAD_POOL_MAX="thread_pool_max=5120"
THREAD_POOL_TIMEOUT="thread_pool_timeout=10"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
LRU_INTERVAL="lru_interval=20"
LISTEN_DEPTH="listen_depth=1024"
                                                                                                                                                                                                                                                                                                                                                                                                                                                                               
#
# # DAEMON_OPTS is used by the init script.  If you add or remove options, make
# # sure you update this section, too.
DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
             -f ${VARNISH_VCL_CONF} \
             -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
             -t ${VARNISH_TTL} \
             -w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \
             -u nobody -g nogroup \
             -s ${VARNISH_STORAGE} \
             -p ${HTTP_RESP_HDR_LEN} \
             -p ${HTTP_MAX_HDR} \
             -p ${HTTP_REQ_HDR_LEN} \
             -p ${THREAD_POOLS} \
             -p ${THREAD_POOL_MIN} \
             -p ${THREAD_POOL_MAX} \
             -p ${THREAD_POOL_TIMEOUT} \
             -p ${LRU_INTERVAL} \
             -p ${LISTEN_DEPTH}"

-----------------------------------------------------------------------------------------------------------------------------------------------

七、Varnish 配置重載腳本

# vim /usr/local/varnish/sbin/varnish_reload_vcl

#!/bin/bash
#
# reload vcl revisited
# A script that loads new vcl based on data from /usr/local/varnish/etc/varnish.conf
# Ingvar Hagelund <ingvar@redpill-linpro.com>
#
# This is free software, distributed under the standard 2 clause BSD license,
# see the LICENSE file in the Varnish documentation directory
#
# The following environment variables have to be set:
# RELOAD_VCL, VARNISH_VCL_CONF, VARNISH_ADMIN_LISTEN_PORT
# The following are optional:
# VARNISH_SECRET_FILE, VARNISH_ADMIN_LISTEN_ADDRESS
#
# Requires GNU bash and GNU date
#
                                                                                                                                                                                                                                                                                                                                                                                                                                          
debug=false
                                                                                                                                                                                                                                                                                                                                                                                                                                          
missing() {
    echo "Missing configuration variable: $1"
    exit 2
}
                                                                                                                                                                                                                                                                                                                                                                                                                                          
print_debug() {
    echo "
Parsed configuration:
RELOAD_VCL=\"$RELOAD_VCL\"
VARNISH_VCL_CONF=\"$VARNISH_VCL_CONF\"
VARNISH_ADMIN_LISTEN_ADDRESS=\"$VARNISH_ADMIN_LISTEN_ADDRESS\"
VARNISH_ADMIN_LISTEN_PORT=\"$VARNISH_ADMIN_LISTEN_PORT\"
VARNISH_SECRET_FILE=\"$VARNISH_SECRET_FILE\"
"
}
                                                                                                                                                                                                                                                                                                                                                                                                                                          
# Read configuration
. /usr/local/varnish/etc/varnish.conf
                                                                                                                                                                                                                                                                                                                                                                                                                                          
$debug && print_debug
                                                                                                                                                                                                                                                                                                                                                                                                                                          
# Check configuration
if [ ! "$RELOAD_VCL" = "1" ]; then
    echo "Error: RELOAD_VCL is not set to 1"
    exit 2
                                                                                                                                                                                                                                                                                                                                                                                                                                          
elif [ -z "$VARNISH_VCL_CONF" ]; then
    echo "Error: VARNISH_VCL_CONF is not set"
    exit 2
                                                                                                                                                                                                                                                                                                                                                                                                                                          
elif [ ! -s "$VARNISH_VCL_CONF" ]; then
    echo "Eror: VCL config $VARNISH_VCL_CONF is unreadable or empty"
    exit 2
                                                                                                                                                                                                                                                                                                                                                                                                                                          
elif [ -z "$VARNISH_ADMIN_LISTEN_ADDRESS" ]; then
    echo "Warning: VARNISH_ADMIN_LISTEN_ADDRESS is not set, using 127.0.0.1"
    VARNISH_ADMIN_LISTEN_ADDRESS="127.0.0.1"
                                                                                                                                                                                                                                                                                                                                                                                                                                          
elif [ -z "$VARNISH_ADMIN_LISTEN_PORT" ]; then
    echo "Error: VARNISH_ADMIN_LISTEN_PORT is not set"
    exit 2
                                                                                                                                                                                                                                                                                                                                                                                                                                          
#elif [ -z "$VARNISH_SECRET_FILE" ]; then
#   echo "Warning: VARNISH_SECRET_FILE is not set"
#   secret=""
                                                                                                                                                                                                                                                                                                                                                                                                                                          
#elif [ ! -s "$VARNISH_SECRET_FILE" ]; then
#   echo "Error: varnish secret file $VARNISH_SECRET_FILE is unreadable or empty"
#   exit 2
else
#   secret="-S $VARNISH_SECRET_FILE"
    echo
fi
                                                                                                                                                                                                                                                                                                                                                                                                                                          
# Done parsing, set up command
#VARNISHADM="varnishadm $secret -T $VARNISH_ADMIN_LISTEN_ADDRESS:$VARNISH_ADMIN_LISTEN_PORT"
VARNISHADM="/usr/local/varnish/bin/varnishadm -T $VARNISH_ADMIN_LISTEN_ADDRESS:$VARNISH_ADMIN_LISTEN_PORT"
                                                                                                                                                                                                                                                                                                                                                                                                                                          
# Now do the real work
new_config="reload_$(date +%FT%H:%M:%S)"
                                                                                                                                                                                                                                                                                                                                                                                                                                          
# Check if we are able to connect at all
if $VARNISHADM vcl.list > /dev/null; then
    $debug && echo vcl.list succeeded
else
    echo "Unable to run $VARNISHADM vcl.list"
    exit 1
fi
                                                                                                                                                                                                                                                                                                                                                                                                                                          
if $VARNISHADM vcl.list | awk ' { print $3 } ' | grep -q $new_config; then
    echo Trying to use new config $new_config, but that is already in use
    exit 2
fi
                                                                                                                                                                                                                                                                                                                                                                                                                                          
current_config=$( $VARNISHADM vcl.list | awk ' /^active/ { print $3 } ' )
                                                                                                                                                                                                                                                                                                                                                                                                                                          
echo "Loading vcl from $VARNISH_VCL_CONF"
echo "Current running config name is $current_config"
echo "Using new config name $new_config"
                                                                                                                                                                                                                                                                                                                                                                                                                                          
if $VARNISHADM vcl.load $new_config $VARNISH_VCL_CONF; then
    $debug && echo "$VARNISHADM vcl.load succeded"
else
    echo "$VARNISHADM vcl.load failed"
    exit 1
fi
                                                                                                                                                                                                                                                                                                                                                                                                                                          
if $VARNISHADM vcl.use $new_config; then
    $debug && echo "$VARNISHADM vcl.use succeded"
else
    echo "$VARNISHADM vcl.use failed"
    exit 1
fi
                                                                                                                                                                                                                                                                                                                                                                                                                                          
$VARNISHADM vcl.list
echo Done
                                                                                                                                                                                                                                                                                                                                                                                                                                          
exit 0

-----------------------------------------------------------------------------------------------------------------------------------------------

八、Varnish啓動腳本

# vim /etc/init.d/varnishd

#!/bin/bash
#
# varnish Control the Varnish Cache
#
# chkconfig: - 90 10
# description: Varnish is a high-perfomance HTTP accelerator
# processname: varnishd
# config: /usr/local/varnish/etc/varnish.conf
# PIDFILE: /var/run/varnishd.pid
#
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
                                                                                                                                                                                                                                                                                                                                                                                                              
BINFILE="/usr/local/varnish/sbin/varnishd"
PROG="varnishd"
                                                                                                                                                                                                                                                                                                                                                                                                              
RETVAL=0
                                                                                                                                                                                                                                                                                                                                                                                                              
GFILE="/usr/local/varnish/etc/varnish.conf"
PIDFILE="/var/run/varnish.pid"
LOCKFILE="/var/lock/varnish.lock"
RELOAD_EXEC="/usr/local/varnish/sbin/varnish_reload_vcl"
                                                                                                                                                                                                                                                                                                                                                                                                              
[[ -e ${GFILE} ]] && . ${GFILE}
                                                                                                                                                                                                                                                                                                                                                                                                              
start() {
    IS_EXIST=`ps -A -oppid,pid,cmd | grep sbin/${PROG} | grep -v grep`
    [[ -n "${IS_EXIST}" ]] && echo "The process of ${PROG} has been running." && exit 1
                                                                                                                                                                                                                                                                                                                                                                                                                 
    [[ ! -x ${BINFILE} ]] && echo ${BINFILE} has no found && exit 5
    [[ ! -f ${GFILE} ]] && echo ${GFILE} has no found && exit 6
                                                                                                                                                                                                                                                                                                                                                                                                                 
    echo -n "Starting Varnish Cache......"
                                                                                                                                                                                                                                                                                                                                                                                                                 
    ulimit -n ${NFILES:-131072}
    ulimit -l ${MEMLOCK:-82000}
    ulimit -u ${NPROCS:-unlimited}
                                                                                                                                                                                                                                                                                                                                                                                                                 
    if [[ "${DAEMON_OPTS}X" == "X" ]]; then
        echo -n "Please setting DAEMON_OPTS options in ${GFILE}"
        RETVAL=6
    else
        VARNISH_CACHE_DIR=`dirname $VARNISH_STORAGE_FILE`
        RETVAL=`grep -w '^VARNISH_STORAGE' ${GFILE} | grep malloc`
        if [[ "${RETVAL}X" = "X" ]]; then
            mkdir -p ${VARNISH_CACHE_DIR} && chown -R nobody:nogroup ${VARNISH_CACHE_DIR}
        else
            [[ -e ${VARNISH_CACHE_DIR} ]] && rm -rf ${VARNISH_CACHE_DIR}
        fi
                                                                                                                                                                                                                                                                                                                                                                                                                     
        ${BINFILE} -P ${PIDFILE} $DAEMON_OPTS >/dev/null 2>&1
        RETVAL=$?
        echo
        [[ $RETVAL -eq 0 ]] && touch ${LOCKFILE}
    fi
                                                                                                                                                                                                                                                                                                                                                                                                                 
    return $RETVAL
}
                                                                                                                                                                                                                                                                                                                                                                                                              
stop() {
    echo -n "Stopping Varnish Cache......"
    /sbin/killproc -QUIT ${PROG}
    RETVAL=$?
    echo
    [[ $RETVAL -eq 0 ]] && rm -f ${LOCKFILE}
                                                                                                                                                                                                                                                                                                                                                                                                                 
    return $RETVAL
}
                                                                                                                                                                                                                                                                                                                                                                                                              
restart() {
    stop
    sleep 1
    start
}
                                                                                                                                                                                                                                                                                                                                                                                                              
reload() {
    [[ "$RELOAD_VCL" = "1" ]] && $RELOAD_EXEC || restart
                                                                                                                                                                                                                                                                                                                                                                                                                 
    RETVAL=$?
    echo
                                                                                                                                                                                                                                                                                                                                                                                                                 
    return $RETVAL
}
                                                                                                                                                                                                                                                                                                                                                                                                              
configtest() {
    if [[ -f ${GFILE} ]]; then
        ${BINFILE} -f ${GFILE} -C -n /tmp >/dev/null 2>&1
        RETVAL=$?
        [[ $? -eq 0 ]] && echo "The syntax is ok." || echo "The syntax is error."
    else
        RETVAL=$?
        echo "The config file: ${GFILE} is no exists."
    fi
                                                                                                                                                                                                                                                                                                                                                                                                                 
    return $RETVAL
}
                                                                                                                                                                                                                                                                                                                                                                                                              
case "$1" in
start|stop|restart|reload|configtest)
    $1
    ;;
                                                                                                                                                                                                                                                                                                                                                                                                                 
*)
    echo "Usage: $0 {start|stop|restart|reload|configtest}"
    exit 2
esac
                                                                                                                                                                                                                                                                                                                                                                                                              
exit $RETVAL

# chmod +x /etc/init.d/varnishd

# chkconfig --add varnishd

-----------------------------------------------------------------------------------------------------------------------------------------------

九、Varnish服務健康檢查腳本

# vim /usr/local/varnish/sbin/check_varnish_health.sh

#!/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin
                                                                                                                                                                                                                                                                                                                                                                                                       
## varnish啓動參數配置文件
VARNISH_CONFIG="/usr/local/varnish/etc/varnish.conf"
                                                                                                                                                                                                                                                                                                                                                                                                       
IS_DEBUG=false
                                                                                                                                                                                                                                                                                                                                                                                                       
[[ -e ${VARNISH_CONFIG} ]] && . ${VARNISH_CONFIG}
                                                                                                                                                                                                                                                                                                                                                                                                       
${IS_DEBUG} && echo "監聽地址:${VARNISH_LISTEN_ADDRESS}  監聽端口:${VARNISH_LISTEN_PORT}"
LAN_IPADDR=`/sbin/ifconfig eth1 | awk -F ':' '/inet addr/{print $2}' | sed 's/[a-zA-Z ]//g'`
                                                                                                                                                                                                                                                                                                                                                                                                       
RETVAL=`nmap --system-dns -sT -p ${VARNISH_LISTEN_PORT} ${LAN_IPADDR} | grep open`
[[ -z ${RETVAL} ]] && /sbin/service varnishd restart >/dev/null 2>&1

# crontab -e

*/5 * * * * /usr/local/varnish/sbin/check_varnish_health.sh >/dev/null 2>&1

相關文章
相關標籤/搜索