Details
-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
Morpheus, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.0.5, 7.1.0, 7.1.1, 7.1.2, 7.1.3, 7.1.5, 7.2.2
-
None
-
Untriaged
-
0
-
No
Description
The changing of the mapping rules results into a rest api call as below which fails with the error in question
2024-04-05T17:41:28.705Z INFO GOXDCR.AdminPort: doChangeReplicationSettingsRequest
|
2024-04-05T17:41:28.706Z INFO GOXDCR.AdminPort: Request params: replicationId=e03798078ccac13b8653a8c9235b05f3/onea/onea_hist justValidate=false includeWarnings=false
|
2024-04-05T17:41:28.706Z ERRO GOXDCR.AdminPort: Validation error in inputs. errorsMap=map[colMappingRules:onea_tca.TradesMvwapBucketed : the specific source namespace does not exist]
|
For validation, the it attempts to get the collection info which is pulled from a cache if it is available. This is see at the code https://src.couchbase.org/source/xref/7.2.2/goproj/src/github.com/couchbase/goxdcr/metadata_svc/collections_manifest_service.go#813
The cache (a.sourceCache) ideally should get updated periodically in a goroutine. But this goroutine is stuck as shown in the stack trace below. The func (*CollectionsManifestAgent).runPeriodicRefresh is executed in background in a loop which fetches the collection/scope info and updates the cache
1 @ 0x43f4b6 0x40b88e 0x40b43d 0xa6e8e5 0xa6658e 0xa64cb3 0xa64725 0xa6356d 0xa632cf 0x98a93c 0x98b80b 0x98b6b5 0x98b9b6 0x471e01# 0xa6e8e4 github.com/couchbase/goxdcr/backfill_manager.(*BackfillRequestHandler).handleBackfillRequestWithArgs+0x104 /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/backfill_manager/backfill_request_handler.go:385# 0xa6658d github.com/couchbase/goxdcr/backfill_manager.(*BackfillMgr).raiseBackfillReq+0x10d /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/backfill_manager/backfill_manager.go:1346# 0xa64cb2 github.com/couchbase/goxdcr/backfill_manager.(*BackfillMgr).diffManifestsAndRaiseBackfill+0x4b2 /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/backfill_manager/backfill_manager.go:1140# 0xa64724 github.com/couchbase/goxdcr/backfill_manager.(*BackfillMgr).handleSrcAndTgtChanges+0x2e4 /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/backfill_manager/backfill_manager.go:1107# 0xa6356c github.com/couchbase/goxdcr/backfill_manager.(*BackfillMgr).handleManifestsChanges+0x10c /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/backfill_manager/backfill_manager.go:937# 0xa632ce github.com/couchbase/goxdcr/backfill_manager.(*BackfillMgr).collectionsManifestChangeCb+0x3ee /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/backfill_manager/backfill_manager.go:916# 0x98a93b github.com/couchbase/goxdcr/metadata_svc.(*CollectionsManifestService).metadataChangeCb+0x27b /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/collections_manifest_service.go:313# 0x98b80a github.com/couchbase/goxdcr/metadata_svc.(*CollectionsManifestAgent).notifyManifestsChange+0x12a /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/collections_manifest_service.go:489# 0x98b6b4 github.com/couchbase/goxdcr/metadata_svc.(*CollectionsManifestAgent).refreshAndNotify+0xb4 /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/collections_manifest_service.go:478# 0x98b9b5 github.com/couchbase/goxdcr/metadata_svc.(*CollectionsManifestAgent).runPeriodicRefresh+0x95 /home/couchbase/jenkins/workspace/couchbase-server-unix/goproj/src/github.com/couchbase/goxdcr/metadata_svc/collections_manifest_service.go:508
|
The reason why this goroutine is stuck while enqueuing into a channel. The goroutine which is supposed to listen to this channel is not running. The reason is because before it could spawn, the function spawning it bailed out due to an error. This is seen at the code https://src.couchbase.org/source/xref/7.2.2/goproj/src/github.com/couchbase/goxdcr/backfill_manager/backfill_request_handler.go#172
I am stating with a strong probability that the it bailed out at this error (which is unable to subscribe to local feed) https://src.couchbase.org/source/xref/7.2.2/goproj/src/github.com/couchbase/goxdcr/backfill_manager/backfill_request_handler.go#167
Attachments
Issue Links
- backports to
-
-
- Open
-