td-agent 收集日誌到kafka的配置

<source>
@type tail
path /data1/td-agent/wechat-tag*.log
tag wechat-tag
read_from_head true
format none
pos_file /var/log/td-agent/wechat-tag.pos
</source>

<match wechat-tag>
@type kafka_buffered
brokers myserver.com:9092
buffer_type file
buffer_path /var/log/td-agent/buffer/wechat-tag
flush_interval 10s
output_data_type attr: message       
</match>

 

解釋說明：

topic: wechat-tag

broker:myserver.com:9092

pos_file 爲了記錄消費文件的位置，防止重啓tdagent之後重複消費

output_data_type：只發送message字段裏的內容