使用Docker鏡像部署ELK日誌系統

使用Docker部署elasticsearch、logstash、kibana

指定版本:6.7.1 (建議使用同一的版本、屏蔽三個軟件間的不兼容性)node

下載鏡像:

docker pull elasticsearch:6.7.1
docker pull logstash:6.7.1
docker pull kibana:6.7.1
 

修改vm.max_map_count

vim /etc/sysctl.conf
添加配置:vm.max_map_count=262144
執行命令,確保生效配置生效: sysctl -p
依據服務器配置而定
 

es集羣

這裏es集羣用了3個節點,配置文件放在 /root/es/config/ 目錄下,docker

注意:es是不容許使用root用戶啓動的,/root/es/config/  目錄最好改爲 777權限,json

es1.yml配置文件內容:vim

cluster.name: elasticsearch-cluster
node.name: es-node1
network.bind_host: 0.0.0.0
network.publish_host: 10.90.101.48
http.port: 9200
transport.tcp.port: 9300
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: true 
node.data: true 
discovery.zen.ping.unicast.hosts: ["10.90.101.48:9300","10.90.101.48:9301","10.90.101.48:9302"]
discovery.zen.minimum_master_nodes: 2
xpack.ml.enabled: false
xpack.monitoring.enabled: false
xpack.security.enabled: false
xpack.watcher.enabled: false
View Code

啓動當前配置文件的es命令:瀏覽器

  docker run -e ES_JAVA_OPTS="-Xms512m -Xmx512m" -d -p 9200:9200 -p 9300:9300 -v /root/es/config/es1.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /root/es/data1:/usr/share/elasticsearch/data --name ES01 elasticsearch:6.7.1

 

es2.yml配置文件內容:
cluster.name: elasticsearch-cluster
node.name: es-node2
network.bind_host: 0.0.0.0
network.publish_host: 10.90.101.48
http.port: 9201
transport.tcp.port: 9301
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: true 
node.data: true 
discovery.zen.ping.unicast.hosts: ["10.90.101.48:9300","10.90.101.48:9301","10.90.101.48:9302"]
discovery.zen.minimum_master_nodes: 2
xpack.ml.enabled: false
xpack.monitoring.enabled: false
xpack.security.enabled: false
xpack.watcher.enabled: false
View Code

啓動當前配置文件的es命令:服務器

  docker run -e ES_JAVA_OPTS="-Xms512m -Xmx512m" -d -p 9201:9201 -p 9301:9301 -v /root/es/config/es2.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /root/es/data2:/usr/share/elasticsearch/data --name ES02 elasticsearch:6.7.1
 
 
es3.yml配置文件內容:
cluster.name: elasticsearch-cluster
node.name: es-node3
network.bind_host: 0.0.0.0
network.publish_host: 10.90.101.48
http.port: 9202
transport.tcp.port: 9302
http.cors.enabled: true
http.cors.allow-origin: "*"
node.master: true 
node.data: true 
discovery.zen.ping.unicast.hosts: ["10.90.101.48:9300","10.90.101.48:9301","10.90.101.48:9302"]
discovery.zen.minimum_master_nodes: 2
xpack.ml.enabled: false
xpack.monitoring.enabled: false
xpack.security.enabled: false
xpack.watcher.enabled: false
View Code

啓動當前配置文件的es命令:app

  docker run -e ES_JAVA_OPTS="-Xms512m -Xmx512m" -d -p 9202:9202 -p 9302:9302 -v /root/es/config/es3.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /root/es/data3:/usr/share/elasticsearch/data --name ES03 elasticsearch:6.7.1
 
 
一、-e JAVA_OPTS="-Xms512m -Xmx512m" 是配置當前es使用jvm的最大內存,內存配置太低會致使CPU很是高,若是服務器內存很大可配置2~4g
二、-p 9200:9200 -p 9300:9300  9200是es提供給外部的通信端口,9300是es節點之間的通信端口
三、 -v /root/es/config/es1.yml:/usr/share/elasticsearch/config/elasticsearch.yml  指定啓動的配置文件
四、-v /root/es/data1:/usr/share/elasticsearch/data 指定es的數據掛載到當前服務器的目錄
五、--name ES01  當前鏡像的名字
 

kibana

kibana.yml配置文件內容:
server.name: kibana
server.host: "0"
elasticsearch.url: http://10.90.101.48:9300
xpack.monitoring.ui.container.elasticsearch.enabled: false
xpack.security.enabled: false
xpack.ml.enabled: false
xpack.monitoring.enabled: false
View Code

啓動kibanacors

  docker run --name kibana -v /root/kibana/config:/usr/share/kibana/config -p 5601:5601 -d kibana:6.7.1
 

logstash

把images中的配置文件拷貝出來:
1. 先運行一個logstash實例
2. docker cp 容器id:/usr/share/logstash/config /root/logstash/config 
    docker cp 容器id:/usr/share/logstash/pipeline /root/logstash/pipeline

pipeline/logstash.confjvm

input{
    http{
        host => "0.0.0.0"
        port => 5050
        additional_codecs => {"application/json"=>"json"}
        codec => "plain"
        threads => 4
        ssl => false
    }
}
output {
    elasticsearch {
        hosts => ["http://10.90.101.48:9200","http://10.90.101.51:9201","http://10.90.101.51:9202"]
        index => "log_%{logtype}_%{+YYYY.MM.dd}"
    }
}
View Code

設置日誌輸入輸出方式elasticsearch

 

config/logstash.yml

http.host: "0.0.0.0"
xpack.monitoring.enabled: false
View Code

啓動logstash

  docker run -d --name logstash -p 5050:5050 -v /root/logstash/config:/usr/share/logstash/config -v /root/logstash/pipeline:/usr/share/logstash/pipeline logstash:6.7.1
 
 

cerebro 一個管理es的工具

docker pull lmenezes/cerebro

啓動cerebro

  docker run --name es-head -p 9000:9000 -d lmenezes/cerebro
 
瀏覽器中打開  ip:9000 連接 http://ip:9200 便可看到es 集羣的狀態
相關文章
相關標籤/搜索