英文:
TCP connection to ActiveMQ on kubernetes keep changing
问题
我设置了如下的场景:
Web应用程序(部署在虚拟机上)连接到ActiveMQ(版本:5.15.10,部署在k8s容器上)。Kubernetes集群包含2个工作节点。我在ActiveMQ的连接仪表板上发现连接在几分钟后发生了变化。
该应用程序抛出以下日志:
javax.jms.JMSException: java.io.EOFException
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:54)
at org.apache.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1960)
at org.apache.activemq.ActiveMQConnection.onException(ActiveMQConnection.java:1979)
at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:114)
at org.apache.activemq.transport.ResponseCorrelator.onException(ResponseCorrelator.java:126)
at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:114)
at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:114)
at org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:173)
at org.apache.activemq.transport.AbstractInactivityMonitor.onException(AbstractInactivityMonitor.java:345)
at org.apache.activemq.transport.TransportSupport.onException(TransportSupport.java:96)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:219)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.EOFException
at java.base/java.io.DataInputStream.readInt(DataInputStream.java:397)
at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:268)
at org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:240)
at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:232)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
... 1 more
从ActiveMQ的一侧,我找到了以下日志:
2020-04-07 15:06:09,575 | WARN | Transport Connection to: tcp://10.171.159.210:41910 failed: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too (>30000) long: tcp://10.171.159.210:41910 | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ InactivityMonitor Worker
Web应用程序以以下OpenWire格式的TCP连接连接到ActiveMQ:
tcp://10.171.159.210:61616?connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0
有没有人有解决方案来保持连接被重用?因为当套接字连接断开时,可能会导致消息被重新传递。
编辑:
我的服务YAML如下:
apiVersion: v1
kind: Service
metadata:
annotations:
service.kubernetes.io/ibm-load-balancer-cloud-provider-ip-type: private
service.kubernetes.io/ibm-load-balancer-cloud-provider-vlan: "2760930"
creationTimestamp: "2020-02-18T06:13:01Z"
labels:
run: message-broker
name: message-broker-service
namespace: default
resourceVersion: "9923438"
selfLink: /api/v1/namespaces/default/services/message-broker-service
uid: b765e641-5215-11ea-8a3b-6eead9e5000e
spec:
clusterIP: 172.21.166.249
externalTrafficPolicy: Cluster
loadBalancerIP: 10.93.197.99
ports:
- name: port8161
nodePort: 32577
port: 8161
protocol: TCP
targetPort: 8161
- name: port61616
nodePort: 31958
port: 61616
protocol: TCP
targetPort: 61616
- name: port61613
nodePort: 31200
port: 61613
protocol: TCP
targetPort: 61613
selector:
run: message-broker
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer:
ingress:
- ip: 10.93.197.99
英文:
I setup an scenario as following:
Web Application (deployed on VM) connects to ActiveMQ (version: 5.15.10 deployed on k8s container). The kubernetes cluster contains 2 worker nodes. I found on connection dashboard of ActiveMQ that the connection changed after some minutes.
The application throws following logs:
javax.jms.JMSException: java.io.EOFException
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:54)
at org.apache.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1960)
at org.apache.activemq.ActiveMQConnection.onException(ActiveMQConnection.java:1979)
at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:114)
at org.apache.activemq.transport.ResponseCorrelator.onException(ResponseCorrelator.java:126)
at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:114)
at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:114)
at org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:173)
at org.apache.activemq.transport.AbstractInactivityMonitor.onException(AbstractInactivityMonitor.java:345)
at org.apache.activemq.transport.TransportSupport.onException(TransportSupport.java:96)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:219)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.EOFException
at java.base/java.io.DataInputStream.readInt(DataInputStream.java:397)
at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:268)
at org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:240)
at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:232)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
... 1 more
From the ActiveMQ side, I find the logs:
2020-04-07 15:06:09,575 | WARN | Transport Connection to: tcp://10.171.159.210:41910 failed: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too (>30000) long: tcp://10.171.159.210:41910 | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ InactivityMonitor Worker
The web application has following tcp connection in openwire format to ActiveMQ:
tcp://10.171.159.210:61616?connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0
Could any one has solution to keep the connection is reused? Because, when the socket connection is broken, it could cause the message is re-delivered.
EDIT
My service yaml is:
apiVersion: v1
kind: Service
metadata:
annotations:
service.kubernetes.io/ibm-load-balancer-cloud-provider-ip-type: private
service.kubernetes.io/ibm-load-balancer-cloud-provider-vlan: "2760930"
creationTimestamp: "2020-02-18T06:13:01Z"
labels:
run: message-broker
name: message-broker-service
namespace: default
resourceVersion: "9923438"
selfLink: /api/v1/namespaces/default/services/message-broker-service
uid: b765e641-5215-11ea-8a3b-6eead9e5000e
spec:
clusterIP: 172.21.166.249
externalTrafficPolicy: Cluster
loadBalancerIP: 10.93.197.99
ports:
- name: port8161
nodePort: 32577
port: 8161
protocol: TCP
targetPort: 8161
- name: port61616
nodePort: 31958
port: 61616
protocol: TCP
targetPort: 61616
- name: port61613
nodePort: 31200
port: 61613
protocol: TCP
targetPort: 61613
selector:
run: message-broker
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer:
ingress:
- ip: 10.93.197.99
专注分享java语言的经验与见解,让所有开发者获益!
评论