Kafka 消費者相關配置

消費者相關配置類爲  org.apache.kafka.clients.consumer.ConsumerConfig

具備如下配置參數

1. GROUP_ID_CONFIG = "group.id";

   消費者分組ID，分組內的消費者只能消費該消息一次，不一樣分組內的消費者能夠重複消費該消息。簡單講就是一條消息會被髮送到不一樣的分組，每一個分組是否消費該消息不會互相影響，可是，分組內的消息只能被其中一個消費者消費一次。Kafka利用這個分組來實現單播和多播的功能。

2. MAX_POLL_RECORDS_CONFIG = "max.poll.records";
　　消費者每次poll數據時的最大數量。

3. MAX_POLL_INTERVAL_MS_CONFIG = "max.poll.interval.ms";
The maximum delay between invocations of poll() when using consumer group management. This places an upper bound on the amount of time that the consumer can be idle before fetching more records. If poll() is not called before expiration of this timeout, then the consumer is considered failed and the group will rebalance in order to reassign the partitions to another member. ";

4. SESSION_TIMEOUT_MS_CONFIG = "session.timeout.ms"

會話響應的時間，超過這個時間kafka能夠選擇放棄消費或者消費下一條消息

"The timeout used to detect consumer failures when using Kafka's group management facility. The consumer sends periodic heartbeats to indicate its liveness to the broker. If no heartbeats are received by the broker before the expiration of this session timeout, then the broker will remove this consumer from the group and initiate a rebalance. Note that the value must be in the allowable range as configured in the broker configuration by <code>group.min.session.timeout.ms</code> and <code>group.max.session.timeout.ms</code>.";5. HEARTBEAT_INTERVAL_MS_CONFIG = "heartbeat.interval.ms";

The expected time between heartbeats to the consumer coordinator when using Kafka's group management facilities. Heartbeats are used to ensure that the consumer's session stays active and to facilitate rebalancing when new consumers join or leave the group. The value must be set lower than <code>session.timeout.ms</code>, but typically should be set no higher than 1/3 of that value. It can be adjusted even lower to control the expected time for normal rebalances.";

6. BOOTSTRAP_SERVERS_CONFIG = "bootstrap.servers";
broker 地址 "host1:port1,host2:port2",-->"localhost:9092,localhost:9093";

7. CLIENT_DNS_LOOKUP_CONFIG = "client.dns.lookup";

8. ENABLE_AUTO_COMMIT_CONFIG = "enable.auto.commit";
　　爲true則自動提交偏移量

9. AUTO_COMMIT_INTERVAL_MS_CONFIG = "auto.commit.interval.ms";
自動提交偏移量週期

10. PARTITION_ASSIGNMENT_STRATEGY_CONFIG = "partition.assignment.strategy";
The class name of the partition assignment strategy that the client will use to distribute partition ownership amongst consumer instances when group management is used";

11. AUTO_OFFSET_RESET_CONFIG = "auto.offset.reset";
重置消費偏移量策略
當偏移量沒有初始值時或者參數非法時，好比數據被刪除時的重置策略。
earliest：automatically reset the offset to the earliest offset
latest：automatically reset the offset to the latest offset
其餘：拋出異常

 <ul><li>earliest: <li>latest: </li><li>none: throw exception to the consumer if no previous offset is found for the consumer's group</li><li>anything else: throw exception to the consumer.</li></ul>";12. FETCH_MIN_BYTES_CONFIG = "fetch.min.bytes";The minimum amount of data the server should return for a fetch request. If insufficient data is available the request will wait for that much data to accumulate before answering the request. The default setting of 1 byte means that fetch requests are answered as soon as a single byte of data is available or the fetch request times out waiting for data to arrive. Setting this to something greater than 1 will cause the server to wait for larger amounts of data to accumulate which can improve server throughput a bit at the cost of some additional latency.";13. FETCH_MAX_BYTES_CONFIG = "fetch.max.bytes";The maximum amount of data the server should return for a fetch request. Records are fetched in batches by the consumer, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that the consumer can make progress. As such, this is not a absolute maximum. The maximum record batch size accepted by the broker is defined via <code>message.max.bytes</code> (broker config) or <code>max.message.bytes</code> (topic config). Note that the consumer performs multiple fetches in parallel.";14. DEFAULT_FETCH_MAX_BYTES = 52428800;15. FETCH_MAX_WAIT_MS_CONFIG = "fetch.max.wait.ms";The maximum amount of time the server will block before answering the fetch request if there isn't sufficient data to immediately satisfy the requirement given by fetch.min.bytes.";16. METADATA_MAX_AGE_CONFIG = "metadata.max.age.ms";17. MAX_PARTITION_FETCH_BYTES_CONFIG = "max.partition.fetch.bytes";The maximum amount of data per-partition the server will return. Records are fetched in batches by the consumer. If the first record batch in the first non-empty partition of the fetch is larger than this limit, the batch will still be returned to ensure that the consumer can make progress. The maximum record batch size accepted by the broker is defined via <code>message.max.bytes</code> (broker config) or <code>max.message.bytes</code> (topic config). See fetch.max.bytes for limiting the consumer request size.";18. DEFAULT_MAX_PARTITION_FETCH_BYTES = 1048576;19. SEND_BUFFER_CONFIG = "send.buffer.bytes";20. RECEIVE_BUFFER_CONFIG = "receive.buffer.bytes";21. CLIENT_ID_CONFIG = "client.id";22. RECONNECT_BACKOFF_MS_CONFIG = "reconnect.backoff.ms";23. RECONNECT_BACKOFF_MAX_MS_CONFIG = "reconnect.backoff.max.ms";24. RETRY_BACKOFF_MS_CONFIG = "retry.backoff.ms";25. METRICS_SAMPLE_WINDOW_MS_CONFIG = "metrics.sample.window.ms";26. METRICS_NUM_SAMPLES_CONFIG = "metrics.num.samples";27. METRICS_RECORDING_LEVEL_CONFIG = "metrics.recording.level";28. METRIC_REPORTER_CLASSES_CONFIG = "metric.reporters";29. CHECK_CRCS_CONFIG = "check.crcs";Automatically check the CRC32 of the records consumed. This ensures no on-the-wire or on-disk corruption to the messages occurred. This check adds some overhead, so it may be disabled in cases seeking extreme performance.";30. KEY_DESERIALIZER_CLASS_CONFIG = "key.deserializer";KEY序列化類配置31. VALUE_DESERIALIZER_CLASS_CONFIG = "value.deserializer";VALUE序列化類配置32. CONNECTIONS_MAX_IDLE_MS_CONFIG = "connections.max.idle.ms";33. String REQUEST_TIMEOUT_MS_CONFIG = "request.timeout.ms";REQUEST_TIMEOUT_MS_DOC = "The configuration controls the maximum amount of time the client will wait for the response of a request. If the response is not received before the timeout elapses the client will resend the request if necessary or fail the request if retries are exhausted.";34. DEFAULT_API_TIMEOUT_MS_CONFIG = "default.api.timeout.ms";Specifies the timeout (in milliseconds) for consumer APIs that could block. This configuration is used as the default timeout for all consumer operations that do not explicitly accept a <code>timeout</code> parameter.";35. INTERCEPTOR_CLASSES_CONFIG = "interceptor.classes";A list of classes to use as interceptors. Implementing the <code>org.apache.kafka.clients.consumer.ConsumerInterceptor</code> interface allows you to intercept (and possibly mutate) records received by the consumer. By default, there are no interceptors.";36. EXCLUDE_INTERNAL_TOPICS_CONFIG = "exclude.internal.topics";Whether records from internal topics (such as offsets) should be exposed to the consumer. If set to <code>true</code> the only way to receive records from an internal topic is subscribing to it.";37. DEFAULT_EXCLUDE_INTERNAL_TOPICS = true;38. LEAVE_GROUP_ON_CLOSE_CONFIG = "internal.leave.group.on.close";39. ISOLATION_LEVEL_CONFIG = "isolation.level";<p>Controls how to read messages written transactionally. If set to <code>read_committed</code>, consumer.poll() will only return transactional messages which have been committed. If set to <code>read_uncommitted</code>' (the default), consumer.poll() will return all messages, even transactional messages which have been aborted. Non-transactional messages will be returned unconditionally in either mode.</p> <p>Messages will always be returned in offset order. Hence, in <code>read_committed</code> mode, consumer.poll() will only return messages up to the last stable offset (LSO), which is the one less than the offset of the first open transaction. In particular any messages appearing after messages belonging to ongoing transactions will be withheld until the relevant transaction has been completed. As a result, <code>read_committed</code> consumers will not be able to read up to the high watermark when there are in flight transactions.</p><p> Further, when in <code>read_committed</code> the seekToEnd method will return the LSO";40. DEFAULT_ISOLATION_LEVEL;