TCP连接到运行在Kubernetes上的ActiveMQ不断变化。

huangapple 未分类评论51阅读模式
英文:

TCP connection to ActiveMQ on kubernetes keep changing

问题

我设置了如下的场景:

Web应用程序(部署在虚拟机上)连接到ActiveMQ(版本:5.15.10,部署在k8s容器上)。Kubernetes集群包含2个工作节点。我在ActiveMQ的连接仪表板上发现连接在几分钟后发生了变化。

该应用程序抛出以下日志:

javax.jms.JMSException: java.io.EOFException
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:54)
at org.apache.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1960)
at org.apache.activemq.ActiveMQConnection.onException(ActiveMQConnection.java:1979)
at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:114)
at org.apache.activemq.transport.ResponseCorrelator.onException(ResponseCorrelator.java:126)
at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:114)
at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:114)
at org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:173)
at org.apache.activemq.transport.AbstractInactivityMonitor.onException(AbstractInactivityMonitor.java:345)
at org.apache.activemq.transport.TransportSupport.onException(TransportSupport.java:96)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:219)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.EOFException
	at java.base/java.io.DataInputStream.readInt(DataInputStream.java:397)
	at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:268)
	at org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:240)
	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:232)
	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
	... 1 more

从ActiveMQ的一侧,我找到了以下日志:

2020-04-07 15:06:09,575 | WARN  | Transport Connection to: tcp://10.171.159.210:41910 failed: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too (>30000) long: tcp://10.171.159.210:41910 | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ InactivityMonitor Worker

Web应用程序以以下OpenWire格式的TCP连接连接到ActiveMQ:

tcp://10.171.159.210:61616?connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0

有没有人有解决方案来保持连接被重用?因为当套接字连接断开时,可能会导致消息被重新传递。

编辑:
我的服务YAML如下:

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.kubernetes.io/ibm-load-balancer-cloud-provider-ip-type: private
    service.kubernetes.io/ibm-load-balancer-cloud-provider-vlan: "2760930"
  creationTimestamp: "2020-02-18T06:13:01Z"
  labels:
    run: message-broker
  name: message-broker-service
  namespace: default
  resourceVersion: "9923438"
  selfLink: /api/v1/namespaces/default/services/message-broker-service
  uid: b765e641-5215-11ea-8a3b-6eead9e5000e
spec:
  clusterIP: 172.21.166.249
  externalTrafficPolicy: Cluster
  loadBalancerIP: 10.93.197.99
  ports:
  - name: port8161
    nodePort: 32577
    port: 8161
    protocol: TCP
    targetPort: 8161
  - name: port61616
    nodePort: 31958
    port: 61616
    protocol: TCP
    targetPort: 61616
  - name: port61613
    nodePort: 31200
    port: 61613
    protocol: TCP
    targetPort: 61613
  selector:
    run: message-broker
  sessionAffinity: None
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - ip: 10.93.197.99
英文:

I setup an scenario as following:
Web Application (deployed on VM) connects to ActiveMQ (version: 5.15.10 deployed on k8s container). The kubernetes cluster contains 2 worker nodes. I found on connection dashboard of ActiveMQ that the connection changed after some minutes.

The application throws following logs:

javax.jms.JMSException: java.io.EOFException
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:54)
at org.apache.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1960)
at org.apache.activemq.ActiveMQConnection.onException(ActiveMQConnection.java:1979)
at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:114)
at org.apache.activemq.transport.ResponseCorrelator.onException(ResponseCorrelator.java:126)
at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:114)
at org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:114)
at org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:173)
at org.apache.activemq.transport.AbstractInactivityMonitor.onException(AbstractInactivityMonitor.java:345)
at org.apache.activemq.transport.TransportSupport.onException(TransportSupport.java:96)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:219)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.io.EOFException
	at java.base/java.io.DataInputStream.readInt(DataInputStream.java:397)
	at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:268)
	at org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:240)
	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:232)
	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
	... 1 more

From the ActiveMQ side, I find the logs:

2020-04-07 15:06:09,575 | WARN  | Transport Connection to: tcp://10.171.159.210:41910 failed: org.apache.activemq.transport.InactivityIOException: Channel was inactive for too (>30000) long: tcp://10.171.159.210:41910 | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ InactivityMonitor Worker

The web application has following tcp connection in openwire format to ActiveMQ:

tcp://10.171.159.210:61616?connectionTimeout=0&keepAlive=true&useInactivityMonitor=false&wireFormat.maxInactivityDuration=0

Could any one has solution to keep the connection is reused? Because, when the socket connection is broken, it could cause the message is re-delivered.

EDIT
My service yaml is:

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.kubernetes.io/ibm-load-balancer-cloud-provider-ip-type: private
    service.kubernetes.io/ibm-load-balancer-cloud-provider-vlan: "2760930"
  creationTimestamp: "2020-02-18T06:13:01Z"
  labels:
    run: message-broker
  name: message-broker-service
  namespace: default
  resourceVersion: "9923438"
  selfLink: /api/v1/namespaces/default/services/message-broker-service
  uid: b765e641-5215-11ea-8a3b-6eead9e5000e
spec:
  clusterIP: 172.21.166.249
  externalTrafficPolicy: Cluster
  loadBalancerIP: 10.93.197.99
  ports:
  - name: port8161
    nodePort: 32577
    port: 8161
    protocol: TCP
    targetPort: 8161
  - name: port61616
    nodePort: 31958
    port: 61616
    protocol: TCP
    targetPort: 61616
  - name: port61613
    nodePort: 31200
    port: 61613
    protocol: TCP
    targetPort: 61613
  selector:
    run: message-broker
  sessionAffinity: None
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - ip: 10.93.197.99

huangapple
  • 本文由 发表于 2020年4月8日 16:07:07
  • 转载请务必保留本文链接:https://java.coder-hub.com/61096066.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定