kafka監控

kafka-web-console

https://github.com/claudemamo/kafka-web-console

部署sbt：

http://www.scala-sbt.org/0.13/tutorial/Manual-Installation.html

http://www.scala-sbt.org/release/tutorial/zh-cn/Installing-sbt-on-Linux.html

KafkaOffsetMonitor

https://github.com/quantifind/KafkaOffsetMonitor/releases/tag/v0.2.0

 

java -cp KafkaOffsetMonitor-assembly-0.2.0.jar com.quantifind.kafka.offsetapp.OffsetGetterWeb --zk localhost:12181 --port 8080 --refresh 5.minutes --retain 1.day

kafka 監控之Mx4jLoader

源碼未作任何修改，在lib下添加jar包mx4j-tools-3.0.1

啓動server以後訪問 ip:8082

/**

  * If mx4j-tools is in the classpath call maybeLoad to load the HTTP interface of mx4j.

  *

  * The default port is 8082. To override that provide e.g. -Dmx4jport=8083

  * The default listen address is 0.0.0.0. To override that provide -Dmx4jaddress=127.0.0.1

  * This feature must be enabled with -Dmx4jenable=true

  *

  * This is a Scala port of org.apache.cassandra.utils.Mx4jTool written by Ran Tavory for CASSANDRA-1068

  * */

JMX監控指標參數列表以下:

 

參數	Mbean名稱	說明
Message in rate	"kafka.server":name="AllTopicsMessagesInPerSec",type="BrokerTopicMetrics"	全部topic消息(進出)流量
Byte in rate	"kafka.server":name="AllTopicsBytesInPerSec",type="BrokerTopicMetrics"
Request rate	"kafka.network":name="{Produce|Fetch-consumer|Fetch-follower}-RequestsPerSec",type="RequestMetrics"
Byte out rate	"kafka.server":name="AllTopicsBytesOutPerSec",type="BrokerTopicMetrics"
Log flush rate and time	"kafka.log":name="LogFlushRateAndTimeMs",type="LogFlushStats"
# of under replicated partitions (|ISR| < |all replicas|)	"kafka.server":name="UnderReplicatedPartitions",type="ReplicaManager"	0
Is controller active on broker	"kafka.controller":name="ActiveControllerCount",type="KafkaController"	only one broker in the cluster should have 1
Leader election rate	"kafka.controller":name="LeaderElectionRateAndTimeMs",type="ControllerStats"	non-zero when there are broker failures
Unclean leader election rate	"kafka.controller":name="UncleanLeaderElectionsPerSec",type="ControllerStats"	0
Partition counts	"kafka.server":name="PartitionCount",type="ReplicaManager"	mostly even across brokers
Leader replica counts	"kafka.server":name="LeaderCount",type="ReplicaManager"	mostly even across brokers
ISR shrink rate	"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager"	If a broker goes down, ISR for some of the partitions will shrink. When that broker is up again, ISR will be expanded once the replicas are fully caught up. Other than that, the expected value for both ISR shrink rate and expansion rate is 0.
ISR expansion rate	"kafka.server":name="ISRExpandsPerSec",type="ReplicaManager"	See above
Max lag in messages btw follower and leader replicas	"kafka.server":name="([-.\w]+)-MaxLag",type="ReplicaFetcherManager"	副本消息滯後數量
Lag in messages per follower replica	"kafka.server":name="([-.\w]+)-ConsumerLag",type="FetcherLagMetrics"	副本消息滯後數量
Requests waiting in the producer purgatory	"kafka.server":name="PurgatorySize",type="ProducerRequestPurgatory"
Requests waiting in the fetch purgatory	"kafka.server":name="PurgatorySize",type="FetchRequestPurgatory"
Request total time	"kafka.network":name="{Produce|Fetch-Consumer|Fetch-Follower}-TotalTimeMs",type="RequestMetrics"
Time the request waiting in the request queue	"kafka.network":name="{Produce|Fetch-Consumer|Fetch-Follower}-QueueTimeMs",type="RequestMetrics"
Time the request being processed at the leader	"kafka.network":name="{Produce|Fetch-Consumer|Fetch-Follower}-LocalTimeMs",type="RequestMetrics"
Time the request waits for the follower	"kafka.network":name="{Produce|Fetch-Consumer|Fetch-Follower}-RemoteTimeMs",type="RequestMetrics"
Time to send the response	"kafka.network":name="{Produce|Fetch-Consumer|Fetch-Follower}-ResponseSendTimeMs",type="RequestMetrics"
Number of messages the consumer lags behind the producer by	"kafka.consumer":name="([-.\w]+)-MaxLag",type="ConsumerFetcherManager"