ELKstack-基於java工程tomcat應用日誌處理過程-02

logstash如何從redis拉出日誌並寫入es？

input {

redis {

batch_count => 1

host => "sz-a-xxxxxredis01-redis-xen.xxxxx.com"

type => "syslog"

data_type => "list"

key => "logstash:syslog-log"

}

redis {

batch_count => 1

host => "sz-a-xxxxxredis01-redis-xen.xxxxx.com"

type => "log4j-json-web"

data_type => "list"

key => "logstash:xxxxxweb-web-log"

}

redis {

batch_count => 1

host => "sz-a-xxxxxredis01-redis-xen.xxxxx.com"

type => "log4j-json-tomcat"

data_type => "list"

key => "logstash:xxxxxweb-tomcat-log"

}

}

output{

if [type] == "syslog" {

elasticsearch { hosts => ["sz-a-xxxxxes01-es-hw.xxxxx.com"]

index => "dev-system-logs-%{+YYYY.MM.dd}"

document_type => "system-logs" }

}

if [type] == "log4j-json-web" {

elasticsearch { hosts => ["sz-a-xxxxxes01-es-hw.xxxxx.com"]

index => "dev-xxxxxweb-master-logs-%{+YYYY.MM.dd}"

document_type => "%{class}" }

}

if [type] == "log4j-json-tomcat" {

elasticsearch { hosts => ["sz-a-xxxxxes01-es-hw.xxxxx.com"]

index => "dev-xxxxxweb-master-logs-%{+YYYY.MM.dd}"

document_type => "tomcat_logs" }

}

}

以上配置中還涉及到了rsyslog日誌的傳輸，借鑑了另外一位大神的配置，他的博客地址：http://dl528888.blog.51cto.com

總結

本文中最後其實少了kibana效果展現，是由於展現圖還在完善中，在後面的記述中，會展現一些kibana呈現的效果。