架構師之路----一步步搭建日誌監控系統（Spring+SpringMVC+Redis+ELK）

時間 2019-11-18

標籤架構師之路步步搭建日誌監控系統 spring+springmvc+redis+elk spring springmvc redis elk 欄目 Spring 简体版

原文原文鏈接

原文地址

http://www.javashuo.com/article/p-xovsexil-et.html前端

技術選型

spring
spring mvc
redis
logback
ELK

開發web工程搭建

經過maven建立一個基於spring mvc的web工程，啓動以後成功顯示hello world（過程略）。nginx

pom文件

<!--spring-->
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-test</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <!--spring-->
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-beans</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
            <version>${spring.version}</version>
        </dependency>

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
            <version>${spring.version}</version>
        </dependency>

        <!--Spring Web + Spring MVC-->
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-webmvc</artifactId>
            <version>${spring.version}</version>
        </dependency>

構建成功

引入logback進行日誌輸出

本第二天志組件使用的是logback；git

pom文件

<!--logback-->
        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>1.1.3</version>
        </dependency>
        <dependency>
            <groupId>org.logback-extensions</groupId>
            <artifactId>logback-ext-spring</artifactId>
            <version>0.1.2</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>jcl-over-slf4j</artifactId>
            <version>1.7.12</version>
        </dependency>

        <!--logstash begin -->
        <dependency>
            <groupId>com.cwbase</groupId>
            <artifactId>logback-redis-appender</artifactId>
            <version>1.1.5</version>
        </dependency>

logback.xml

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <appender name="LOGSTASH" class="com.cwbase.logback.RedisAppender">
        <source>SpringELK</source>
        <type>dev-type</type>
        <tags>dev</tags>
        <host>127.0.0.1</host>
        <port>6379</port>
        <key>logstash</key>
    </appender>

    <root level="DEBUG">
        <appender-ref ref="LOGSTASH" />
    </root>
</configuration>

因爲項目比較小，採用了日誌寫入redis，大型項目可參考將redis替換爲kafka。商業項目可用logstash監聽nginx access log，定時將日誌上傳kafka集羣。github

如log4j將日誌發送到logstash：

#輸出日誌到logstash
log4j.appender.logstash=org.apache.log4j.net.SocketAppender
log4j.appender.logstash.RemoteHost=127.0.0.1
log4j.appender.logstash.port=4560
log4j.appender.logstash.ReconnectionDelay=60000
log4j.appender.logstash.LocationInfo=true

logstash input配置：

input {
    log4j {
        host => "127.0.0.1"
        port => 4560
    }
}

logstash有兩個做用：

前端logstash作日誌收集代理，做用同flume；
後端logstash作解析日誌，鏈接es創建索引；

日誌輸出

private Logger log = LoggerFactory.getLogger(HomeController.class);

    @RequestMapping(value = "/hello", method = RequestMethod.GET)
    public String hello() {
        log.error("Main錯誤");
        return "/home/hello";
    }