春季Kafka与SSL,生产者错误“重复键”

huangapple 未分类评论51阅读模式
英文:

Spring kafka with ssl, producer error "Duplicate key"

问题

我有一个带有SSL的Kafka服务器。在向其生产消息时,出现错误:

2020-05-29 19:19:27.677 [kafka-producer-network-thread | producer-1] ERROR o.a.k.c.producer.internals.Sender - [Producer clientId=producer-1] Uncaught error in Kafka producer I/O thread:
java.lang.IllegalStateException: Duplicate key host:9091 (id: 1 rack: null)
	at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133)
	at java.util.HashMap.merge(HashMap.java:1254)
	at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320)

我对它所暗示的“Duplicate key”感到困惑。我的application.properties配置如下:

spring.kafka.bootstrap-servers=host:9091
spring.kafka.properties.security.protocol=SSL
spring.kafka.ssl.trust-store-password=kafka
spring.kafka.properties.ssl.endpoint.identification.algorithm=
spring.kafka.ssl.key-store-password=abc
spring.kafka.ssl.trust-store-location=file:/path/client.truststore.jks
spring.kafka.ssl.key-store-location=file:/path/client.keystore.jks

运行时日志中Kafka生产者的部分配置:

    acks = 1
    bootstrap.servers = [host:9091]
    client.dns.lookup = default
    client.id =
    enable.idempotence = false
    interceptor.classes = []
    key.serializer = class org.apache.kafka.common.serialization.StringSerializer
    linger.ms = 0
    partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
    security.protocol = SSL
    ssl.cipher.suites = null
    ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
    ssl.endpoint.identification.algorithm =
    ssl.key.password = null
    ssl.keymanager.algorithm = SunX509
    ssl.keystore.location = /path/client.keystore.jks
    ssl.keystore.password = [hidden]
    ssl.keystore.type = JKS
    ssl.protocol = TLS
    ssl.provider = null
    ssl.secure.random.implementation = null
    ssl.trustmanager.algorithm = PKIX
    ssl.truststore.location = /path/client.truststore.jks
    ssl.truststore.password = [hidden]
    ssl.truststore.type = JKS
    transactional.id = null
    value.serializer = class org.apache.kafka.common.serialization.StringSerializer

有什么地方出错了吗?任何帮助将不胜感激。

我认为根据此文档中的表格,版本可能不兼容。我的版本如下:

spring-boot: 2.2.6.RELEASE
spring-kafka: 2.3.7.RELEASE
Kafka服务器:0.10.1.1

然而,我能够在没有SSL的情况下,通过使用当前版本成功地从Kafka服务器(0.10.0.0)生产和消费数据。

英文:

I have a Kafka server with SSL. When producing messages to it, error occurs:

2020-05-29 19:19:27.677 [kafka-producer-network-thread | producer-1] ERROR o.a.k.c.producer.internals.Sender - [Producer clientId=producer-1] Uncaught error in kafka producer I/O thread: 
java.lang.IllegalStateException: Duplicate key host:9091 (id: 1 rack: null)
	at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133)
	at java.util.HashMap.merge(HashMap.java:1254)
	at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320)

I'm confused about "Duplicate key" it implies. My configuration in application.properties:

spring.kafka.bootstrap-servers=host:9091
spring.kafka.properties.security.protocol=SSL
spring.kafka.ssl.trust-store-password=kafka
spring.kafka.properties.ssl.endpoint.identification.algorithm=
spring.kafka.ssl.key-store-password=abc
spring.kafka.ssl.trust-store-location=file:/path/client.truststore.jks
spring.kafka.ssl.key-store-location=file:/path/client.keystore.jks

Part configuration of Kafka producer from logs during runtime:

    acks = 1
    bootstrap.servers = [host:9091]
    client.dns.lookup = default
	client.id = 
	enable.idempotence = false
	interceptor.classes = []
	key.serializer = class org.apache.kafka.common.serialization.StringSerializer
	linger.ms = 0
	partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
	security.protocol = SSL
	ssl.cipher.suites = null
	ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
	ssl.endpoint.identification.algorithm = 
	ssl.key.password = null
	ssl.keymanager.algorithm = SunX509
	ssl.keystore.location = /path/client.keystore.jks
	ssl.keystore.password = [hidden]
	ssl.keystore.type = JKS
	ssl.protocol = TLS
	ssl.provider = null
	ssl.secure.random.implementation = null
	ssl.trustmanager.algorithm = PKIX
	ssl.truststore.location = /path/client.truststore.jks
	ssl.truststore.password = [hidden]
	ssl.truststore.type = JKS
	transactional.id = null
	value.serializer = class org.apache.kafka.common.serialization.StringSerializer

Is there anything wrong? Any help will be appreciated.

It comes to me that versions may be incompatible according to the table in this document. Versions of mine:

spring-boot: 2.2.6.RELEASE
spring-kafka: 2.3.7.RELEASE
Kafka server: 0.10.1.1

However, I was able to successfully produce and consume from Kafka server (0.10.0.0) without SSL by utilizing the current version.

huangapple
  • 本文由 发表于 2020年5月29日 19:35:56
  • 转载请务必保留本文链接:https://java.coder-hub.com/62085067.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定