springboot+redis+ELK（ElasticSearch+Logstash+kibana）

上一篇已經搭建好elk的環境

如今只要搭建一個elk+redis的環境

使用redis是是防止大批量日誌的時候logstash沒法及時處理

安裝redis

    wget http://download.redis.io/releases/redis-3.2.8.tar.gz

解壓

    tar -zxvf redis-3.2.8.tar.gz

移動

    mv redis-3.2.8.tar.gz /usr/local

修改redis配置文件

    vim /usr/local/redis-3.2.8.tar.gz/redis.conf

protected-mode：修改成no,不然redis只有本機才能訪問

執行make編譯Redis：
make MALLOC=libc
注意：make命令執行完成編譯後，會在src目錄下生成6個可執行文件，分別是redis-server、redis-cli、redis-benchmark、redis-check-aof、redis-check-rdb、redis-sentinel。

啓動redis

    ./src/redis-server redis.conf

搭建redis和logstash
（1）編輯logstash的logstash.conf文件，輸入端修改成redis

input {
        redis {
                data_type => "list"
                type => "redis-input"
                key => "logstash:redis"
                host => "192.168.42.128"
                port => 6379
                threads => 5
                codec => "json"
        }
}
output {
        elasticsearch {
                hosts => "192.168.212.37:9201"
                index => "logstash-test"
        }
        stdout {
                codec => rubydebug {}
        }
}

host改爲本身的redis服務器地址便可。

從新啓動logstash

    ./bin/logstash  -f /usr/local/logstash-5.6.3/etc/logstash.conf

下面新建一個springboot demo測試下

pom.xml    

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
   <groupId>com.cwbase</groupId>
   <artifactId>logback-redis-appender</artifactId>
   <version>1.1.5</version>
</dependency>

新建logback.xml

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <appender name="Console" class="ch.qos.logback.core.ConsoleAppender">
        <Target>System.out</Target>
        <encoder>
            <pattern>[%d{HH:mm:ss}][%t][%p][%c]-%m%n</pattern>
        </encoder>
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>info</level>
        </filter>
    </appender>
    <appender name="LOGSTASH" class="com.cwbase.logback.RedisAppender">
        <source>logstashdemo</source>
        <type>dev</type>
        <host>192.168.42.128</host>
        <key>logstash:redis</key>
        <tags>dev</tags>
        <mdc>true</mdc>
        <location>true</location>
        <callerStackIndex>0</callerStackIndex>
    </appender>

    <root level="info">
        <appender-ref ref="Console"/>
        <appender-ref ref="LOGSTASH"/>
    </root>
</configuration>

DemoApplication:

@SpringBootApplication
@RestController
public class DemoApplication {
    private static final Logger logger = Logger.getLogger(DemoApplication.class);

    @GetMapping("/")
    public String getTestString() {
        logger.info("哈哈哈哈哈哈哈哈哈哈哈哈哈");
        logger.error("錯錯錯錯錯錯錯錯錯錯錯錯錯錯錯錯");
        return "this is my first log";
    }

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }
}

運行程序 localhost:8080

控制檯

logstash控制檯

瀏覽器打開kibana  http://192.168.42.128:5601

自行進行條件篩選和搜索