elasticsearch7.x集羣安裝（含head、bigdesk、kibana插件）

網址：https://www.elastic.co

192.168.14.239 es-node1
192.168.14.240 es-node2
192.168.14.241 es-node3

=====> 初始化

① 關閉防火牆、selinux
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g'  /etc/selinux/config
setenforce 0
systemctl stop firewalld
systemctl disable firewalld

② 修改系統最大打開文件數和進程數
cat <<EOF >> /etc/security/limits.conf

* soft nofile 65536
* hard nofile 65536
* soft nproc 2048
* hard nproc 4096
EOF
echo vm.max_map_count=655360 >> /etc/sysctl.conf
sysctl -p

③ 配置主機名及互信
cat <<EOF >> /etc/hosts
192.168.14.239 es-node1
192.168.14.240 es-node2
192.168.14.241 es-node3
EOF
hostname es-node1
hostnamectl set-hostname es-node1
ssh-keygen
ssh-copy-id es-node1
ssh-copy-id es-node2
ssh-copy-id es-node3

④ 配置yum源
yum -y install wget vim
cd /etc/yum.repos.d/
mkdir backup
mv *.repo backup
# 阿里雲yun源
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
yum clean all
yum makecache
# epel源
yum -y install epel-release      
yum clean all
yum makecache

# elk源
cat <<EOF > /etc/yum.repos.d/elk.repo
[elk]
name=elk
baseurl=https://mirrors.tuna.tsinghua.edu.cn/elasticstack/yum/elastic-6.x/
enable=1
gpgcheck=0
EOF

⑤ 源碼安裝java
mkdir -p /data/apps/
tar -xf jdk-8u11-linux-x64.tar.gz
mv jdk1.8.0_11/ jdk
cat <<EOF > /etc/profile.d/jdk.sh
JAVA_HOME=/data/apps/jdk
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
PATH=$JAVA_HOME/bin:$PATH
export JAVA_HOME CLASSPATH PATH
EOF
source /etc/profile

=====> 安裝elasticsearch
① 下載
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.3.1-linux-x86_64.tar.gz
tar -xf elasticsearch-7.3.1-linux-x86_64.tar.gz
mv elasticsearch-7.3.1 /data/apps/elasticsearch
cd /data/apps
useradd es
chown -R es.es elasticsearch
su - es
mkdir -pv /home/es/{data,logs}/elastic

② 配置
cd elasticsearch
vim config/elasticsearch.yml     
cluster.name: bigdata      # 集羣名稱 
node.name: node-1       # 節點名稱 
path.data: /home/es/data/elastic   # es索引庫的數據存儲目錄
path.logs: /home/es/logs/elastic   # es進程啓動後，對應的日誌信息存放目錄 
network.host: 0.0.0.0
# 容許跨域請求
http.cors.enabled: true   
http.cors.allow-origin: "*"
http.cors.allow-credentials: true
#discovery.seed_hosts: ["node-1"]
cluster.initial_master_nodes: ["es-node1"]
transport.tcp.port: 9300                 # 節點間交互的tcp端口，默認9300
discovery.zen.minimum_master_nodes: 2    # 防腦裂，集羣中至少又2臺節點可用，不然集羣就癱瘓。計算公式： 節點數/2+1
discovery.zen.ping.unicast.hosts: ['es-node1','es-node2','es-node3'] #

#Running as a daemon
./bin/elasticsearch -d -p pid_file
# shut down Elasticsearch
pkill -F pid_file 
# Checking that Elaelasticsearch is running
curl -XGET 'http://127.0.0.1:9200'

====> ES-Head Plugin 方便對ES進行各類操做的客戶端工具
https://github.com/mobz/elasticsearch-head
*** 插件不能安裝在es的plugin目錄下
git clone git://github.com/mobz/elasticsearch-head.git
cd elasticsearch-head
yum -y install nodejs npm
npm init -f   # 解決 npm WARN enoent ENOENT: no such file or directory, open '/soft/elasticsearch/plugins/package.json'
npm install -g grunt-cli
npm install grunt --save
npm install grunt-contrib-clean
npm install grunt-contrib-concat
npm install grunt-contrib-watch
npm install grunt-contrib-connect
npm install grunt-contrib-copy
npm install phantomjs-prebuilt@2.1.14 --ignore-scripts
npm install grunt-contrib-jasmine

# elasticsearch-head 目錄下的 Gruntfile.js 文件，在 options 屬性內增長 hostname，設置爲 0.0.0.0
connect: {
    server: {
        options: {
            hostname: '0.0.0.0',
            port: 9100,
            base: '.',
            keepalive: true
        }
    }
}
# 修改elasticsearch-head/_site/app.js
this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://node-1:9200";

# 啓動elasticsearch-head
nohup grunt server > /dev/null 2>&1 &

# 屏啓
yum -y install screen
screen -S es-head
grunt server

# 訪問
http://localhost:9100/

====> BigDesk Plugin 實時狀態監控（jvm、linux、elasticsearch的狀況）
https://github.com/lukas-vlcek/bigdesk
mkdir elasticsearch/plugins/bigdesk/_site
unzip bigdesk-master.zip -C plugin/bigdesk/_site
mv plugin/bigdesk/_site/bigdesk-master/* plugins/bigdesk/_site
cd plugin/bigdesk
cat <<EOF > plugin-descriptor.properties
description=bigdesk
version=master
site=true
name=bigdesk
EOF

cd elasticsearch/plugins/bigdesk/_site/js/store
vim  BigdeskStore.js
return (major == 1 && minor >= 0 && maintenance >= 0 && (build != 'Beta1' || build != 'Beta2'));
-->>
return (minor >= 0 && maintenance >= 0 && (build != 'Beta1' || build != 'Beta2'));

#python -m SimpleHTTPServer
nohup python -m SimpleHTTPServer > /dev/null 2>&1 &
#nohup python -m SimpleHTTPServer 8888 > /dev/null 2>&1 &
https://ip:9200/_plugin/bigdesk/

 

====> kibana Plugin 讀取es集羣中索引庫的type信息，並使用可視化的方式呈現

 

wget https://artifacts.elastic.co/downloads/kibana/kibana-7.3.2-linux-x86_64.tar.gz
shasum -a 512 kibana-7.3.2-linux-x86_64.tar.gz
tar -xzf kibana-7.3.2-linux-x86_64.tar.gz
mv kibana-7.3.2-linux-x86_64 kibana
mv kibana /data/apps/es-plugin
cd /data/apps/es-plugin/kibana

vim $kibana/config/kibana.yml
server.port: 5601                                //監聽端口
server.host: "192.168.14.239"                      //監聽IP地址，建議內網ip
elasticsearch.url: "http://192.168.14.239:9200"    //elasticsearch鏈接kibana的URL，也能夠填寫192.168.1.32，由於它們是一個集羣

useradd -s /sbin/nologin kibana
chown -R kibana.kibana kibana
su - kibana
./bin/kibana

192.168.14.239:5601   # 訪問
 

=====> logstash 收集日誌文件內容 #vim /etc/logstash/conf.d/system.conf input {         file {                 path => "/var/log/messages"                     //日誌路徑                 type => "system"                                //定義類型                 start_position => "beginning"                   //表示logstash從頭開始讀取文件內容                 stat_interval => "2"                            //logstash每隔多久檢查一次被監聽文件狀態（是否有更新）,默認是1秒         } } output {         elasticsearch {                 hosts => ["192.168.1.31"]                       //指定hosts                 index => "systemlog-%{+YYYY.MM.dd}"             //指定索引名稱                 } }