Loading...

Details

Type: Bug
Resolution: Fixed
Priority: Critical
Fix Version/s: 7.1.0
Affects Version/s: 7.1.0
Component/s: XDCR
Labels:
- eagle-eye
- system-test
Environment:
Enterprise Edition 7.1.0 build 1361

Triage:
Untriaged
Operating System:
Centos 64-bit
Link to Log File, atop/blg, CBCollectInfo, Core dump:

Hide
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.104.137.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.104.155.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.104.157.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.104.5.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.104.67.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.104.69.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.104.70.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.105.107.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.105.111.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.106.100.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.106.188.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.108.103.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.120.245.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.121.117.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.121.3.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.123.27.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.123.28.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.96.148.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.96.251.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.96.252.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.96.253.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.97.119.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.97.121.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.97.122.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.97.239.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.97.242.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.98.135.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.99.11.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.99.20.zip
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.99.21.zip

Show
https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.104.137.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.104.155.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.104.157.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.104.5.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.104.67.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.104.69.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.104.70.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.105.107.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.105.111.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.106.100.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.106.188.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.108.103.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.120.245.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.121.117.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.121.3.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.123.27.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.123.28.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.96.148.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.96.251.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.96.252.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.96.253.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.97.119.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.97.121.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.97.122.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.97.239.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.97.242.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.98.135.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.99.11.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.99.20.zip https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1632849891/collectinfo-2021-09-28T172454-ns_1%40172.23.99.21.zip
Story Points:
1
Is this a Regression?:
Unknown

Description

Test -

-test tests/integration/neo/test_neo_kv_gsi_coll_xdcr_backup_sgw_fts_itemct_txns_eventing_cbas_scale3_magma.yml -scope tests/integration/neo/scope_neo_with_backup_magma.yml

Day - 2
Cycle - 2
Scale - 3

On 172.23.97.119
diag.log

2021-09-28T08:08:06.822-07:00, ns_log:0:info:message(ns_1@172.23.97.119) - Service 'goxdcr' exited with status 2. Restarting. Messages:

github.com/couchbase/goxdcr/parts.(*dcpStreamReqHelper).getNewVersion(0xc0d28464b0, 0xc1387f0000)

	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:123 +0x227

github.com/couchbase/goxdcr/parts.(*DcpNozzle).startUprStream(0xc138a5e000, 0xc23c010228, 0xc0c4577240, 0x0, 0x0)

	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:1477 +0x67

github.com/couchbase/goxdcr/parts.(*DcpNozzle).startUprStreams_internal(0xc138a5e000, 0xc23c8a5fa0, 0x9, 0x10, 0x10, 0x0)

	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:1339 +0x1ac

github.com/couchbase/goxdcr/parts.(*DcpNozzle).startUprStreams(0xc138a5e000, 0x0, 0x0)

	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:1300 +0x469

created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start

	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:768 +0x3f3

xdcr.log

2021-09-28T08:08:04.958-07:00 ERRO GOXDCR.DcpNozzle: Error: dcpStreamHelper for vbno: 552 internal version overflow

panic: runtime error: invalid memory address or nil pointer dereference

[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0xa6a423]

goroutine 1617458 [running]:

github.com/couchbase/goxdcr/pipeline_svc.(*PipelineSupervisor).OnEvent(0xc1328cce00, 0xc28a0106e0)

	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/pipeline_svc/pipeline_supervisor.go:273 +0x8a3

github.com/couchbase/goxdcr/component.(*AbstractComponent).RaiseEvent(0xc0cb4d3900, 0xc28a0106e0)

	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/component/abstract_component.go:92 +0xf5

github.com/couchbase/goxdcr/parts.(*dcpStreamReqHelper).getNewVersion(0xc0d28464b0, 0xc1387f0000)

	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:123 +0x227

github.com/couchbase/goxdcr/parts.(*DcpNozzle).startUprStream(0xc138a5e000, 0xc23c010228, 0xc0c4577240, 0x0, 0x0)

	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:1477 +0x67

github.com/couchbase/goxdcr/parts.(*DcpNozzle).startUprStreams_internal(0xc138a5e000, 0xc23c8a5fa0, 0x9, 0x10, 0x10, 0x0)

	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:1339 +0x1ac

github.com/couchbase/goxdcr/parts.(*DcpNozzle).startUprStreams(0xc138a5e000, 0x0, 0x0)

	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:1300 +0x469

created by github.com/couchbase/goxdcr/parts.(*DcpNozzle).Start

	/home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/parts/dcp_nozzle.go:768 +0x3f3

Attachments

Issue Links

is caused by

MB-48671 XDCR - DCP Nozzle may not be cleanly shut down

Closed

Gerrit Reviews

- Issue Only
- Show All Reviews
- Show Open Reviews
- Show All Issues
- Show Open Issues

No reviews matched the request. Check your Options in the drop-down menu of this sections header.

[System Test] Service 'goxdcr' exited with status 2.

Details

Description

Attachments

Issue Links

Gerrit Reviews

Activity

People

Dates

Gerrit Reviews

PagerDuty