ELK實戰（Springboot日誌輸出查找）

需求

1. 把分佈式系統，集羣日誌集中處理快速查詢
2. 搭建ELK並與springboot日誌輸出結合

搭建ELK

1. 基於我前面的elasticsearch搭建博客文檔docker-compose.yml基礎上進行添加修改
2. 新建docker-compose.yml文件，內容以下

version: '2'
services:
  elasticsearch-central: 
    image: elasticsearch:5.6.4
    container_name: es1
    volumes:
      - /root/mydocker/docker-es/es1.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - /root/mydocker/docker-es/data1:/usr/share/elasticsearch/data
    restart: always
    environment:
      - ES_CLUSTERNAME=elasticsearch
      - "ES_JAVA_OPTS=-Xmx50m -Xms50m"
    command: elasticsearch
    ports:
      - "9200:9200"
      - "9300:9300"
  elasticsearch-data: 
    image: elasticsearch:5.6.4
    container_name: es2
    volumes:
      - /root/mydocker/docker-es/es2.yml:/usr/share/elasticsearch/config/elasticsearch.yml
      - /root/mydocker/docker-es/data2:/usr/share/elasticsearch/data
    restart: always
    environment:
      - ES_CLUSTERNAME=elasticsearch
      - "ES_JAVA_OPTS=-Xmx50m -Xms50m"
    command: elasticsearch
    ports:
      - "9201:9200"
      - "9301:9300"
    links:
      - elasticsearch-central:elasticsearch
  elasticsearch-head:
    image: mobz/elasticsearch-head:5
    container_name: head
    restart: always
    volumes:
      - /root/mydocker/docker-es/head-conf/Gruntfile.js:/usr/src/app/Gruntfile.js
      - /root/mydocker/docker-es/head-conf/app.js:/usr/src/app/_site/app.js
    ports:
      - "9100:9100"
    links:
      - elasticsearch-central:elasticsearch
  kibana:
    image: kibana
    container_name: kibana
    restart: always
    environment:
      - ELASTICSEARCH_URL=http://ip:9200
    links:
      - elasticsearch-central:elasticsearch
    ports:
      - "5601:5601"
  logstash:
    image: docker.elastic.co/logstash/logstash:5.5.1
    command: logstash -f /etc/logstash/conf.d/logstash.conf
    volumes:
      - $PWD/logstash/conf.d:/etc/logstash/conf.d
      - $PWD/log:/tmp
    container_name: logstash551
    hostname: logstash
    restart: always
    depends_on:
      - elasticsearch-central
    ports:
      - "7001-7005:7001-7005"
      - "4567:4567"

• kibana爲何配置一個地址，參考 https://blog.csdn.net/liukuan73/article/details/52635602?locationNum=3&fps=1

1. 同級目錄新建 logstash/conf.d 目錄用於掛載，建logstash.conf文件，內容以下

input {
  tcp {
    port => 4567
    codec => json_lines
  }
}
 
output {
  elasticsearch {
    action => "index"
    hosts => ["172.16.147.200:9200","172.16.147.200:9201"]
    index => "%{[appname]}"
  }
}

• port表示監聽端口，要與你向外開發的TCP端口一直，也就是數據入口的端口
• codec => json_lines 表示以json數據輸入
• hosts 表示elasticsearch輸入地址
• index 表示es索引名稱
• logstash文檔 https://doc.yonyoucloud.com/doc/logstash-best-practice-cn/get_start/full_config.html

1. 使用 docker-compose up -d 啓動所有容器
2. 訪問ip:5601 進入kibana管理界面，建立索引
3. 建立applog索引的監控，用來用kibana查詢日誌

與springboot結合

1. 在springboot項目中pom加入

<!-- logstash -->
<dependency>
    <groupId>net.logstash.logback</groupId>
    <artifactId>logstash-logback-encoder</artifactId>
    <version>4.11</version>
</dependency>

1. 在logback.xml中加入

<appender name="logstash"
          class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <destination>127.0.0.1:4567</destination>
    <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
        <level>INFO</level>
    </filter>
    <!-- encoder必須配置,有多種可選 -->
    <encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder">
        <customFields>{"appname":"carer"}</customFields>
    </encoder>
    <connectionStrategy>
        <roundRobin>
            <connectionTTL>5 minutes</connectionTTL>
        </roundRobin>
    </connectionStrategy>
</appender>
<!-- 開發環境 -->
<springProfile name="dev">
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>${PATTERN}</pattern>
        </encoder>
    </appender>
    <logger name="com.zhiyis" level="debug"/>
    <root level="info">
        <appender-ref ref="CONSOLE"/>
        <appender-ref ref="logstash" />
    </root>
</springProfile>

• 主要是上半部分，下半部分就是加個<appender-ref ref="logstash" />
• 要修改的就是<destination>127.0.0.1:4567</destination> 這個中把logstash地址填上，端口別弄錯~我就由於開放端口4567，這裏填了網上例子中的4560，明明把logstash配置改爲了4560仍是不通，後來想了想才發現，我配docker容器開放的端口就只有4567
• 把springboot項目運行起來，調幾個測試接口，而後去kibana看日誌
• 更高級的用法再慢慢研究了，左邊一排過濾的能更精準查找，還有表單的統計等等待研究

參考
https://blog.csdn.net/guduyishuai/article/details/79228306
https://www.cnblogs.com/zhyg/p/6994314.html