Xitrum學習筆記19 - 部署到生產環境

能夠直接運行 Xitrum

Browser ------ Xitrum instance

或者在在負載均衡器如HAProxy, 或反向代理如Apache或Nginx以後：

Browser ------ Load balancer/Reverse proxy -+---- Xitrum instance1
                                            +---- Xitrum instance2

打包路徑

運行sbt/sbt xitrum-package來生成target/xitrum路徑中的內容，準備向生產服務器部署

target/xitrum
  config
    [config files]
  public
    [static public files]
  lib
    [dependencies and packaged project file]
  script
    runner
    runner.bat
    scalive
    scalive.jar
    scalive.bat

自定義xitrum-package

默認狀況下，sbt/sbt xitrum-package命令被配置成拷貝路徑config、public和script到target/xitrum。

若是要再拷貝更多的路徑和文件，須要修改build.sbt：

XitrumPackage.copy("config", "public, "script", "doc/README.txt", "etc.")

獲取更多信息，參考 https://github.com/xitrum-framework/xitrum-package

鏈接Scala console運行JVM進程

在沒有預先設置的生產環境中，爲了進行現場調試，可使用Scalive鏈接Scala console來運行JVM進程

在script路徑運行scalive：

script
  runner
  runner.bat
  scalive
  scalive.jar
  scalive.bat

在生產模式下當系統啓動時啓動Xitrum

script/runner (for *nix) and script/runner.bat (for Windows) 是運行帶有main方法的對象的腳本。

用它能夠啓動在生產環境上的web服務器，script/runner quickstart.Boot

能夠修改runner腳原本調整JVM設置

要在Linux系統啓動時在後臺啓動Xitrum，一個簡單的方法是把下面一行加到 /etc/rc.local中

su - user_foo_bar -c /path/to/the/runner/script/above &

daemontools是另外一種解決方法。要在CentOS安裝它，請查閱相關資料。

daemontools資料： http://cr.yp.to/daemontools.html

或使用Supervisord（一個進程控制系統，詳見官網http://supervisord.org/）， /etc/supervisord.conf 示例:

[program:my_app]
directory=/path/to/my_app
command=/path/to/my_app/script/runner quickstart.Boot
autostart=true
autorestart=true
startsecs=3
user=my_user
redirect_stderr=true
stdout_logfile=/path/to/my_app/log/stdout.log
stdout_logfile_maxbytes=10MB
stdout_logfile_backups=7
stdout_capture_maxbytes=1MB

  stdout_events_enabled=false
  environment=PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/aws/bin:~/bin

其餘選擇有：runit、upstart

設置端口

Xitrum默認監聽端口8000和4430，能夠在config/xitrum.conf改變端口。

能夠用如下命令更新/etc/sysconfig/iptables使用80以替代8000、443替代4430

sudo su - root
chmod 700 /etc/sysconfig/iptables
iptables-restore < /etc/sysconfig/iptables
iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8000
iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 443 -j REDIRECT --to-port 4430
iptables -t nat -I OUTPUT -p tcp -d 127.0.0.1 --dport 80 -j REDIRECT --to-ports 8000
iptables -t nat -I OUTPUT -p tcp -d 127.0.0.1 --dport 443 -j REDIRECT --to-ports 4430
iptables-save -c > /etc/sysconfig/iptables
chmod 644 /etc/sysconfig/iptables

若是有其餘進程運行在80和443上，要先中止這些進程

sudo /etc/init.d/httpd stop
sudo chkconfig httpd off

調整linux以應對大量鏈接

參考：

https://docs.basho.com/riak/kv/2.2.3/using/performance/ 

https://docs.basho.com/riak/kv/2.2.3/using/performance/amazon-web-services/

https://www.frozentux.net/ipsysctl-tutorial/chunkyhtml/

https://www.frozentux.net/ipsysctl-tutorial/chunkyhtml/tcpvariables.html

增長文件打開數限制

對於Linux來講，每一個鏈接都當作一個打開的文件。默認最大打開文件數是1024。要增長這個數字，修改/etc/security/limits.conf

* soft nofile 1024000
* hard nofile 1024000

登出在登入才能使以上配置生效，運行 ulimit -n命令查看變動是否生效。

調整kernel內核

根據文章《A Million-user Comet Application with Mochiweb》，修改/etc/sysctl.conf

# General gigabit tuning
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
# This gives the kernel more memory for TCP
# which you need with many (100k+) open socket connections
net.ipv4.tcp_mem = 50576 64768 98152
# Backlog
net.core.netdev_max_backlog = 2048
net.core.somaxconn = 1024
net.ipv4.tcp_max_syn_backlog = 2048
net.ipv4.tcp_syncookies = 1
# If you run clients
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 10

運行sudo sysctl -p應用這個配置，無須重啓，內核應該能夠應對更多的鏈接了

Note about backlog

TCP does the 3-way handshake for making a connection. When a remote client connects to the server, it sends SYN
packet, and the server OS replies with SYN-ACK packet, then again that remote client sends ACK packet and the
connection is established. Xitrum gets the connection when it is completely established.
According to the article Socket backlog tuning for Apache, connection timeout happens because of SYN packet loss
which happens because backlog queue for the web server is filled up with connections sending SYN-ACK to slow
clients.
According to the FreeBSD Handbook, the default value of 128 is typically too low for robust handling of new connections in a heavily loaded web server environment. For such environments, it is recommended to increase this value to
1024 or higher. Large listen queues also do a better job of avoiding Denial of Service (DoS) attacks.
The backlog size of Xitrum is set to 1024 (memcached also uses this value), but you also need to tune the kernel as
above.
To check the backlog config:

cat /proc/sys/net/core/somaxconn

Or:

sysctl net.core.somaxconn

To tune temporarily, you can do like this:

sudo sysctl -w net.core.somaxconn=1024

HAProxy tips

爲SockJS配置HAProxy，查看這個示例

defaults
    mode http
    timeout connect 10s
    timeout client 10h # 客戶端非活動狀態的超時時長Set to long time to avoid WebSocket connections being closed when there's
    timeout server 10h # Set to long time to avoid ERR_INCOMPLETE_CHUNKED_ENCODING on Chrome
frontend xitrum_with_discourse
    bind 0.0.0.0:80
    option forwardfor
    acl is_discourse path_beg /forum
    use_backend discourse if is_discourse
    default_backend xitrum
backend xitrum
    server srv_xitrum 127.0.0.1:8000
backend discourse
    server srv_discourse 127.0.0.1:3000

要想不重啓讓HAProxy從新加載配置文件，參考

https://serverfault.com/questions/165883/is-there-a-way-to-add-more-backend-server-to-haproxy-without-restarting-haproxy

HAProxy比Nginx在使用上更加簡單，Xitrum響應靜態文件更加快速，沒必要使用Nginx的靜態文件功能。

Nginx tips

若是在Xitrum上使用WebSocket或者SockJS功能，而且想在Nginx1.2後運行Xitrum，必須安裝像nginx_tcp_proxy_module同樣的模塊。

Nginx1.3以上的版本直接支持WebSocket。

Nginx爲反向代理默認使用HTTP1.0協議，若是後臺服務器返回的是分塊的響應，你須要讓Nginx使用HTTP1.1：

location / {
  proxy_http_version 1.1;
  proxy_set_header Connection "";
  proxy_pass http://127.0.0.1:8000;
}

此文檔描述如何讓http保持瀏覽器和服務端之間的長鏈接，也要設置

proxy_set_header Connection "";