Details
-
Bug
-
Resolution: Fixed
-
Minor
-
6.0.1
-
Untriaged
-
-
Unknown
-
CX Sprint 148, CX Sprint 150
Description
Build : 6.5.0-2943
Test : -test tests/integration/test_allFeatures_alice_timers.yml -scope tests/integration/scope_Xattrs_Alice.yml
Scale : 3
Iteration : 1st
Log inspection from the system test run shows the following in the Analytics error logs on one of the nodes - 172.23.96.168:
2019-04-15T01:57:57.620-07:00 ERRO CBAS.work.RegisterNodeWork [Worker:ClusterController] node ebf98db4b271d38dad6c76a24cafe597 registration failed
|
org.apache.hyracks.api.exceptions.HyracksException: HYR0010: Node ebf98db4b271d38dad6c76a24cafe597 does not exist
|
at org.apache.hyracks.api.exceptions.HyracksException.create(HyracksException.java:57) ~[hyracks-api.jar:6.5.0-2943]
|
at org.apache.hyracks.control.cc.cluster.NodeManager.addNode(NodeManager.java:106) ~[hyracks-control-cc.jar:6.5.0-2943]
|
at org.apache.hyracks.control.cc.work.RegisterNodeWork.doRun(RegisterNodeWork.java:64) [hyracks-control-cc.jar:6.5.0-2943]
|
at org.apache.hyracks.control.common.work.SynchronizableWork.run(SynchronizableWork.java:43) [hyracks-control-common.jar:6.5.0-2943]
|
at org.apache.hyracks.control.common.work.WorkQueue$WorkerThread.run(WorkQueue.java:127) [hyracks-control-common.jar:6.5.0-2943]
|
2019-04-15T01:57:59.214-07:00 ERRO CBAS.tcp.TCPEndpoint [TCPEndpoint IO Thread [null]] Unexpected tcp io error in connection TCPConnection[Remote Address: /172.23.99.11:9117 Local Address: null]
|
org.apache.hyracks.api.exceptions.NetException: Socket Closed
|
at org.apache.hyracks.net.protocols.muxdemux.MultiplexedConnection.driveReaderStateMachine(MultiplexedConnection.java:360) ~[hyracks-net.jar:6.5.0-2943]
|
at org.apache.hyracks.net.protocols.muxdemux.MultiplexedConnection.notifyIOReady(MultiplexedConnection.java:119) ~[hyracks-net.jar:6.5.0-2943]
|
at org.apache.hyracks.net.protocols.tcp.TCPEndpoint$IOThread.run(TCPEndpoint.java:197) [hyracks-net.jar:6.5.0-2943]
|
2019-04-15T01:57:59.217-07:00 ERRO CBAS.tcp.TCPEndpoint [TCPEndpoint IO Thread [/0.0.0.0:9116]] Unexpected tcp io error in connection TCPConnection[Remote Address: /172.23.99.11:60709 Local Address: /0.0.0.0:9116]
|
org.apache.hyracks.api.exceptions.NetException: Socket Closed
|
at org.apache.hyracks.net.protocols.muxdemux.MultiplexedConnection.driveReaderStateMachine(MultiplexedConnection.java:360) ~[hyracks-net.jar:6.5.0-2943]
|
at org.apache.hyracks.net.protocols.muxdemux.MultiplexedConnection.notifyIOReady(MultiplexedConnection.java:119) ~[hyracks-net.jar:6.5.0-2943]
|
at org.apache.hyracks.net.protocols.tcp.TCPEndpoint$IOThread.run(TCPEndpoint.java:197) [hyracks-net.jar:6.5.0-2943]
|
The test was performing a swap rebalance of analytics nodes :
[2019-04-15T01:55:15-07:00, sequoiatools/couchbase-cli:e16a9d] server-add -c 172.23.108.103:8091 --server-add 172.23.106.188:8091 -u Administrator -p password --server-add-username Administrator --server-add-password password --services analytics
|
[2019-04-15T01:55:30-07:00, sequoiatools/couchbase-cli:ac660b] rebalance -c 172.23.108.103:8091 --server-remove 172.23.99.11 -u Administrator -p password
|
[2019-04-15T01:59:23-07:00, sequoiatools/cmd:a2eca6] 60
|
[2019-04-15T02:00:32-07:00, sequoiatools/cmd:95d7ff] 300
|