计算机基础教程

# Kafka Client Compatibility

Bidirectional Client Compatibility is now supported, you don't need to worry about the compatibility matrix anymore, for KIP-35 enabled clients, any version are good, KIP-35 is released from Broker protocol - 0.10.0, Java clients - 0.10.2 https://stackoverflow.com/questions/55691662/determine-the-kafka-client-compatibility-with-kafka-broker#/

https://cwiki.apache.org/confluence/display/KAFKA/KIP-35+-+Retrieving+protocol+version#/

https://spring.io/projects/spring-kafka org.springframework.kafka org.apache.kafka https://www.cnblogs.com/wangb0402/p/6187796.html

# Versions

# 2.3.0

https://archive.apache.org/dist/kafka/2.3.0/RELEASE_NOTES.html

Kafka 2.3.0 includes a number of significant new features. Here is a summary of some notable changes:

  • There have been several improvements to the Kafka Connect REST API.
  • Kafka Connect now supports incremental cooperative rebalancing.
  • Kafka Streams now supports an in-memory session store and window store.
  • The AdminClient now allows users to determine what operations they are authorized to perform on topics.
  • There is a new broker start time metric.
  • JMXTool can now connect to secured RMI ports.
  • An incremental AlterConfigs API has been added. The old AlterConfigs API has been deprecated.
  • We now track partitions which are under their min ISR count.
  • Consumers can now opt-out of automatic topic creation, even when it is enabled on the broker.
  • Kafka components can now use external configuration stores (KIP-421).
  • We have implemented improved replica fetcher behavior when errors are encountered.

# 2.4.0

https://archive.apache.org/dist/kafka/2.4.0/RELEASE_NOTES.html

Kafka 2.4.0 includes a number of significant new features. Here is a summary of some notable changes:

  • Allow consumers to fetch from closest replica.
  • Support for incremental cooperative rebalancing to the consumer rebalance protocol.
  • MirrorMaker 2.0 (MM2), a new multi-cluster, cross-datacenter replication engine.
  • New Java authorizer Interface.
  • Support for non-key joining in KTable.
  • Administrative API for replica reassignment.

# 2.5.0

https://archive.apache.org/dist/kafka/2.5.0/RELEASE_NOTES.html

Kafka 2.5.0 includes a number of significant new features. Here is a summary of some notable changes:

  • TLS 1.3 support (1.2 is now the default)
  • Co-groups for Kafka Streams
  • Incremental rebalance for Kafka Consumer
  • New metrics for better operational insight
  • Upgrade Zookeeper to 3.5.7
  • Deprecate support for Scala 2.11

# 2.6.0

https://archive.apache.org/dist/kafka/2.6.0/RELEASE_NOTES.html

Kafka 2.6.0 includes a number of significant new features. Here is a summary of some notable changes:

  • TLSv1.3 has been enabled by default for Java 11 or newer
  • Significant performance improvements, especially when the broker has large numbers of partitions
  • Smooth scaling out of Kafka Streams applications
  • Kafka Streams support for emit on change
  • New metrics for better operational insight
  • Kafka Connect can automatically create topics for source connectors when configured to do so
  • Improved error reporting options for sink connectors in Kafka Connect
  • New Filter and conditional SMTs in Kafka Connect
  • The default value for the client.dns.lookup configuration is now use_all_dns_ips
  • Upgrade Zookeeper to 3.5.8

# 2.7.0

https://archive.apache.org/dist/kafka/2.7.0/RELEASE_NOTES.html

Kafka 2.7.0 includes a number of significant new features. Here is a summary of some notable changes:

  • Configurable TCP connection timeout and improve the initial metadata fetch
  • Enforce broker-wide and per-listener connection creation rate (KIP-612, part 1)
  • Throttle Create Topic, Create Partition and Delete Topic Operations
  • Add TRACE-level end-to-end latency metrics to Streams
  • Add Broker-side SCRAM Config API
  • Support PEM format for SSL certificates and private key
  • Add RocksDB Memory Consumption to RocksDB Metrics
  • Add Sliding-Window support for Aggregations
public class PolyProducer<K, V> implements Producer<K, V> {

    @Override
    public void sendOffsetsToTransaction(Map<TopicPartition, OffsetAndMetadata> offsets,
            ConsumerGroupMetadata groupMetadata) throws ProducerFencedException {
        // TODO Auto-generated method stub
        
    }

# Features

# Incremental rebalance

https://cwiki.apache.org/confluence/display/KAFKA/KIP-429%3A+Kafka+Consumer+Incremental+Rebalance+Protocol

explained: https://www.confluent.io/blog/incremental-cooperative-rebalancing-in-kafka/

stop-the-world rebalancing =>

# Upgrade Zookeeper

# Significant performance improvements, especially when the broker has large numbers of partitions

# Configurable TCP connection timeout and improve the initial metadata fetch

https://issues.apache.org/jira/browse/KAFKA-9893

# Enforce broker-wide and per-listener connection creation rate (KIP-612, part 1)

# Throttle Create Topic, Create Partition and Delete Topic Operations

Expose Consumer Group Metadata for Transactional Producer https://issues.apache.org/jira/browse/KAFKA-9383 https://github.com/apache/kafka/pull/7906