flume 示例一收集tomcat日誌

例子場景描述：將tomcat的日誌收集到指定的目錄，tomcat 安裝在/opt/tomcat, 日誌存放在var/log/data。

配置tomcat.conf 以下：

# A single-node Flume configuration

# Name the components on this agent

agent1.sources = source1
agent1.sinks = sink1
agent1.channels = channel1

 

# Describe/configure source1

agent1.sources.source1.type = exec
agent1.sources.source1.command = tail -n +0 -F /opt/tomcat/logs/ catalina.out
agent1.sources.source1.channels = channel1


# Describe sink1
agent1.sinks.sink1.type = file_roll
agent1.sinks.sink1.sink.directory=/var/log/data

# Use a channel which buffers events in memory
agent1.channels.channel1.type = file
agent1.channels.channel1.checkpointDir=/var/checkpoint
agent1.channels.channel1.dataDirs=/var/tmp
agent1.channels.channel1.capacity = 1000
agent1.channels.channel1.transactionCapactiy = 100
 

# Bind the source and sink to the channel
agent1.sources.source1.channels = channel1
agent1.sinks.sink1.channel = channel1

運行agent 

bin/flume-ng agent --conf conf --conf-file tomcat.conf --name agent1 -Dflume.root.logger=INFO,console

agent自動會將日誌收集到指定的var/log/data

注意：hadoop-core-1.0.1.jar 放在flume 的lib目錄中。不然報ClassNotFoundException
經過以上例子能夠推廣到收集apache server 、nginx日誌