Loading...

XML

Word

Printable

Type: Bug
Resolution: Duplicate
Priority: Major
Fix Version/s: 7.6.4
Affects Version/s: 7.6.1
Component/s: analytics
Labels:
- Upgrade
- analytics
- k8s
- kubernetes
- operator
- triaged
- upgrade
Environment:
CAO Image : couchbase/couchbase-operator:2.7.0-arm64
Couchbase Enterprise Edition 7.6.1-3200
Couchbase Enterprise Edition 7.2.5-7596
Environment : Kind Kubernetes environment run locally on Mac

Triage:
Untriaged
Operating System:
Linux x86_64
Link to Log File, atop/blg, CBCollectInfo, Core dump:

Hide
Operator logs:
https://cb-engineering.s3.amazonaws.com/MB-62745/cbopinfo-20240716T182653+0530.tar.gz

Cluster logs:
https://cb-engineering.s3.amazonaws.com/MB-62745/collectinfo-2024-07-16T125636-ns_1%40cb-example-0011.cb-example.default.svc.zip
https://cb-engineering.s3.amazonaws.com/MB-62745/collectinfo-2024-07-16T125636-ns_1%40cb-example-0012.cb-example.default.svc.zip
https://cb-engineering.s3.amazonaws.com/MB-62745/collectinfo-2024-07-16T125636-ns_1%40cb-example-0013.cb-example.default.svc.zip

Show
Operator logs: https://cb-engineering.s3.amazonaws.com/MB-62745/cbopinfo-20240716T182653+0530.tar.gz Cluster logs: https://cb-engineering.s3.amazonaws.com/MB-62745/collectinfo-2024-07-16T125636-ns_1%40cb-example-0011.cb-example.default.svc.zip https://cb-engineering.s3.amazonaws.com/MB-62745/collectinfo-2024-07-16T125636-ns_1%40cb-example-0012.cb-example.default.svc.zip https://cb-engineering.s3.amazonaws.com/MB-62745/collectinfo-2024-07-16T125636-ns_1%40cb-example-0013.cb-example.default.svc.zip
Story Points:
0
Is this a Regression?:
Unknown

Steps to reproduce

Created a 3 node cluster on k8s with operator with all services.
Created a bucket with 0 docs.
Upgraded the cluster from 7.2.5-7596 to 7.6.1-3200.
The last node swap rebalance was failed manually by killing memcached.
A downgrade was issued post that.
Post full cluster downgrade, an upgrade was issued.
Post upgrade the storage backend of the bucket was changed from couchstore to magma.
Service analytics crashes

Service 'cbas' exited with status 2. Restarting. Messages:

main.(*Mgr).areServerGroupsBalancedLOCKED(0x40000ca280)

goproj/src/github.com/couchbase/cbas/cbas/manager.go:1948 +0x80

main.(*Mgr).ensureSeverGroupsLOCKED(0x40000ca280)

goproj/src/github.com/couchbase/cbas/cbas/manager.go:1930 +0x20

main.monitorServerGroups.func1.(*Mgr).serverGroupsVersionUpdated.func1(0x4000563d88?)

goproj/src/github.com/couchbase/cbas/cbas/manager.go:1924 +0x23c

main.(*Mgr).updateStateLOCKED(0x40000ca280, 0x8106b0?)

goproj/src/github.com/couchbase/cbas/cbas/manager.go:1247 +0x34

main.(*Mgr).updateState(0x40000ca280, 0x26?)

goproj/src/github.com/couchbase/cbas/cbas/manager.go:1243 +0x68

main.(*Mgr).serverGroupsVersionUpdated(...)

goproj/src/github.com/couchbase/cbas/cbas/manager.go:1913

main.monitorServerGroups.func1({0x83ae80?, 0x40004bc140?})

goproj/src/github.com/couchbase/cbas/cbas/high_availability.go:86 +0x6c

main.runObserveStreamingEndpoint.func1()

goproj/src/github.com/couchbase/cbas/cbas/compat.go:400 +0x264

created by main.runObserveStreamingEndpoint in goroutine 47

goproj/src/github.com/couchbase/cbas/cbas/compat.go:380 +0x53c

duplicates

MB-62845 after rebalance out of CC, "last orchestrator" tasks are not performed

No reviews matched the request. Check your Options in the drop-down menu of this sections header.

There are no open Gerrit changes

[Analytics][K8S] : Service 'cbas' exited with status 2