[System Test] Service 'goxdcr' exited with status 2.

Build - 7.1.0-1085
Test -

-test tests/integration/neo/test_neo_kv_gsi_coll_xdcr_backup_sgw _fts_itemct_txns_eventing _cbas_scale3_magma.yml -scope tests/integration/neo/scope_neo_with_backup_magma.yml

Scale - 3
Day - 2

On 172.23.120.73

• diag.log

  2021-07-24T00:36:15.158-07:00, ns_log:0:info:message(ns_1@172.23.120.73) - Service 'goxdcr' exited with status 2. Restarting. Messages:

  github.com/couchbase/goxdcr/service_impl.(*BucketTopologySvcWatcher).shouldSendToCh(0xc0000fcc00, 0xc0193c4eb0, 0x4a, 0xc8525b, 0xc, 0x1)

  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/bucketTopologyService.go:1164 +0x85

  github.com/couchbase/goxdcr/service_impl.(*BucketTopologySvcWatcher).updateOnce(0xc0000fcc00, 0xc8525b, 0xc, 0xc0001d3c20)

  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/bucketTopologyService.go:955 +0x285

  github.com/couchbase/goxdcr/service_impl.(*BucketTopologySvcWatcher).run.func1(0x3b9aca00, 0xc0000fcc00, 0xc8525b, 0xc, 0xc0001d3c20, 0xc0001b3fa0)

  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/bucketTopologyService.go:892 +0x254

  created by github.com/couchbase/goxdcr/service_impl.(*BucketTopologySvcWatcher).run

  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/bucketTopologyService.go:876 +0x371

• goxdcr.log

  2021-07-24T00:36:15.042-07:00 INFO GOXDCR.PipelineMgr: Replication Status = map[408030b35137f655c784fa33ad2b65ef/bucket4/bucket4:name={408030b35137f655c784fa33ad2b65ef/bucket4/bucket4}, status={Replicating}, errors={[]}, oldProgress={All incoming nozzles have been opened}, progress={Pipeline is running} 408030b35137f655c784fa33ad2b65ef/bucket8/bucket8:name={408030b35137f655c784fa33ad2b65ef/bucket8/bucket8}, status={Replicating}, errors={[]}, oldProgress={All incoming nozzles have been opened}, progress={Pipeline is running}, oldBackfillProgress={All incoming nozzles have been opened}, backfillProgress={Pipeline is running} 408030b35137f655c784fa33ad2b65ef/bucket9/bucket9:name={408030b35137f655c784fa33ad2b65ef/bucket9/bucket9}, status={Replicating}, errors={[]}, oldProgress={All incoming nozzles have been opened}, progress={Pipeline is running}, oldBackfillProgress={All incoming nozzles have been opened}, backfillProgress={Pipeline is running} 408030b35137f655c784fa33ad2b65ef/default/remote:name={408030b35137f655c784fa33ad2b65ef/default/remote}, status={Replicating}, errors={[]}, oldProgress={All incoming nozzles have been opened}, progress={Pipeline is running}]

  panic: 408030b35137f655c784fa33ad2b65ef/bucket4/bucket4_ReplicationMgrUpdateStats - false not found

  goroutine 81 [running]:

  github.com/couchbase/goxdcr/service_impl.(*BucketTopologySvcWatcher).checkHighSeqnosReceiverAwaitingData(0xc0000fcc00, 0xc0193c4eb0, 0x4a, 0xc001cc1e00, 0xc001cc1ec8)

  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/bucketTopologyService.go:1183 +0x24e

  github.com/couchbase/goxdcr/service_impl.(*BucketTopologySvcWatcher).shouldSendToCh(0xc0000fcc00, 0xc0193c4eb0, 0x4a, 0xc8525b, 0xc, 0x1)

  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/bucketTopologyService.go:1164 +0x85

  github.com/couchbase/goxdcr/service_impl.(*BucketTopologySvcWatcher).updateOnce(0xc0000fcc00, 0xc8525b, 0xc, 0xc0001d3c20)

  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/bucketTopologyService.go:955 +0x285

  github.com/couchbase/goxdcr/service_impl.(*BucketTopologySvcWatcher).run.func1(0x3b9aca00, 0xc0000fcc00, 0xc8525b, 0xc, 0xc0001d3c20, 0xc0001b3fa0)

  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/bucketTopologyService.go:892 +0x254

  created by github.com/couchbase/goxdcr/service_impl.(*BucketTopologySvcWatcher).run

  /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/service_impl/bucketTopologyService.go:876 +0x371