英文:
Spring kafka with ssl, producer error "Duplicate key"
问题
我有一个带有SSL的Kafka服务器。在向其生产消息时,出现错误:
2020-05-29 19:19:27.677 [kafka-producer-network-thread | producer-1] ERROR o.a.k.c.producer.internals.Sender - [Producer clientId=producer-1] Uncaught error in Kafka producer I/O thread:
java.lang.IllegalStateException: Duplicate key host:9091 (id: 1 rack: null)
at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133)
at java.util.HashMap.merge(HashMap.java:1254)
at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320)
我对它所暗示的“Duplicate key”感到困惑。我的application.properties
配置如下:
spring.kafka.bootstrap-servers=host:9091
spring.kafka.properties.security.protocol=SSL
spring.kafka.ssl.trust-store-password=kafka
spring.kafka.properties.ssl.endpoint.identification.algorithm=
spring.kafka.ssl.key-store-password=abc
spring.kafka.ssl.trust-store-location=file:/path/client.truststore.jks
spring.kafka.ssl.key-store-location=file:/path/client.keystore.jks
运行时日志中Kafka生产者的部分配置:
acks = 1
bootstrap.servers = [host:9091]
client.dns.lookup = default
client.id =
enable.idempotence = false
interceptor.classes = []
key.serializer = class org.apache.kafka.common.serialization.StringSerializer
linger.ms = 0
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
security.protocol = SSL
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm =
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = /path/client.keystore.jks
ssl.keystore.password = [hidden]
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = /path/client.truststore.jks
ssl.truststore.password = [hidden]
ssl.truststore.type = JKS
transactional.id = null
value.serializer = class org.apache.kafka.common.serialization.StringSerializer
有什么地方出错了吗?任何帮助将不胜感激。
我认为根据此文档中的表格,版本可能不兼容。我的版本如下:
spring-boot: 2.2.6.RELEASE
spring-kafka: 2.3.7.RELEASE
Kafka服务器:0.10.1.1
然而,我能够在没有SSL的情况下,通过使用当前版本成功地从Kafka服务器(0.10.0.0)生产和消费数据。
英文:
I have a Kafka server with SSL. When producing messages to it, error occurs:
2020-05-29 19:19:27.677 [kafka-producer-network-thread | producer-1] ERROR o.a.k.c.producer.internals.Sender - [Producer clientId=producer-1] Uncaught error in kafka producer I/O thread:
java.lang.IllegalStateException: Duplicate key host:9091 (id: 1 rack: null)
at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133)
at java.util.HashMap.merge(HashMap.java:1254)
at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320)
I'm confused about "Duplicate key" it implies. My configuration in application.properties
:
spring.kafka.bootstrap-servers=host:9091
spring.kafka.properties.security.protocol=SSL
spring.kafka.ssl.trust-store-password=kafka
spring.kafka.properties.ssl.endpoint.identification.algorithm=
spring.kafka.ssl.key-store-password=abc
spring.kafka.ssl.trust-store-location=file:/path/client.truststore.jks
spring.kafka.ssl.key-store-location=file:/path/client.keystore.jks
Part configuration of Kafka producer from logs during runtime:
acks = 1
bootstrap.servers = [host:9091]
client.dns.lookup = default
client.id =
enable.idempotence = false
interceptor.classes = []
key.serializer = class org.apache.kafka.common.serialization.StringSerializer
linger.ms = 0
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
security.protocol = SSL
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm =
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = /path/client.keystore.jks
ssl.keystore.password = [hidden]
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = /path/client.truststore.jks
ssl.truststore.password = [hidden]
ssl.truststore.type = JKS
transactional.id = null
value.serializer = class org.apache.kafka.common.serialization.StringSerializer
Is there anything wrong? Any help will be appreciated.
It comes to me that versions may be incompatible according to the table in this document. Versions of mine:
spring-boot: 2.2.6.RELEASE
spring-kafka: 2.3.7.RELEASE
Kafka server: 0.10.1.1
However, I was able to successfully produce and consume from Kafka server (0.10.0.0) without SSL by utilizing the current version.
专注分享java语言的经验与见解,让所有开发者获益!
评论