Flume NG中的ElasticSearch Sink

  ElasticSearch是一個基於Lucene的搜索服務器。它提供了一個分佈式多用戶能力的全文搜索引擎,基於RESTful web接口。Elasticsearch是用Java開發的,並做爲Apache許可條款下的開放源碼發佈,是當前流行的企業級搜索引擎。設計用於雲計算中,可以達到實時搜索,穩定,可靠,快速,安裝使用方便。web

  在Flume中的ElasticSearchSink支持Flume與Elasticsearch整合,能夠將Flume採集的數據傳輸到Elasticsearch中,其主要配置項以下apache

a1.sources = r1
a1.sinks = k1
a1.channels = c1

# source
......

# sink
a1.sinks.k1.type=org.apache.flume.sink.elasticsearch.ElasticSearchSink  
a1.sinks.k1.batchSize=10000  
a1.sinks.k1.hostNames=192.168.10.83:9300,192.168.10.84:9301  
a1.sinks.k1.indexType = flume_kafka  
a1.sinks.k1.indexName=logstash  
a1.sinks.k1.clusterName=unifyloggingplatform  
a1.sinks.k1.serializer=org.apache.flume.sink.elasticsearch.ElasticSearchLogStashEventSerializer  
a1.sinks.k1.indexNameBuilder=org.apache.flume.sink.elasticsearch.SimpleIndexNameBuilder 

# channel
......

# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
相關文章
相關標籤/搜索