Details
-
Bug
-
Resolution: Fixed
-
Blocker
-
2.6.1
-
None
-
None
-
3 - Droids in Distress
-
1
Description
The failure:
Error Messagepods "test-couchbase-f9zx2-0001" not foundStacktracegoroutine 846 [running]: |
runtime/debug.Stack()
|
runtime/debug/stack.go:24 +0x72 |
github.com/couchbase/couchbase-operator/test/e2e/e2eutil.Die(0xc000f95ba0, {0x6b0b460, 0xc003590640}) |
github.com/couchbase/couchbase-operator/test/e2e/e2eutil/util.go:1433 +0x36 |
github.com/couchbase/couchbase-operator/test/e2e/e2eutil.MustKillPodForMember(0xc000070000?, 0xc0008f42a0, 0xc000070000, 0xc000070000?, 0xa8?) |
github.com/couchbase/couchbase-operator/test/e2e/e2eutil/util.go:1059 +0x126 |
github.com/couchbase/couchbase-operator/test/e2e.TestUpgradeSupportableKillExistingStatefulPodOnRebalance(0xc000f95ba0) |
github.com/couchbase/couchbase-operator/test/e2e/upgrade_test.go:645 +0xcdd |
testing.tRunner(0xc000f95ba0, 0x55f6d10) |
testing/testing.go:1576 +0x217 |
created by testing.(*T).Run
|
testing/testing.go:1629 +0x806 |
There's a rebalance error in the logs:
{"level":"info","ts":"2024-02-02T13:34:22Z","logger":"cluster","msg":"Reconciliation failed","cluster":"test-rf588/test-couchbase-rzq6w","error":"failed to rebalance: timeout: unexpected rebalance error: node test-couchbase-rzq6w-0005 state Down, expected Active","stack":"github.com/couchbase/couchbase-operator/pkg/cluster.(*Cluster).verifyRebalance.func1\n\tgithub.com/couchbase/couchbase-operator/pkg/cluster/rebalance.go:293\ngithub.com/couchbase/couchbase-operator/pkg/util/retryutil.Retry\n\tgithub.com/couchbase/couchbase-operator/pkg/util/retryutil/retryutil.go:14\ngithub.com/couchbase/couchbase-operator/pkg/util/retryutil.RetryFor\n\tgithub.com/couchbase/couchbase-operator/pkg/util/retryutil/retryutil.go:30\ngithub.com/couchbase/couchbase-operator/pkg/cluster.(*Cluster).verifyRebalance\n\tgithub.com/couchbase/couchbase-operator/pkg/cluster/rebalance.go:307\ngithub.com/couchbase/couchbase-operator/pkg/cluster.(*Cluster).rebalance\n\tgithub.com/couchbase/couchbase-operator/pkg/cluster/rebalance.go:246\ngithub.com/couchbase/couchbase-operator/pkg/cluster.(*ReconcileMachine).handleRebalance\n\tgithub.com/couchbase/couchbase-operator/pkg/cluster/nodereconcile.go:1336\ngithub.com/couchbase/couchbase-operator/pkg/cluster.(*ReconcileMachine).exec\n\tgithub.com/couchbase/couchbase-operator/pkg/cluster/nodereconcile.go:313\ngithub.com/couchbase/couchbase-operator/pkg/cluster.(*Cluster).reconcileMembers\n\tgithub.com/couchbase/couchbase-operator/pkg/cluster/reconcile.go:268\ngithub.com/couchbase/couchbase-operator/pkg/cluster.(*Cluster).reconcile\n\tgithub.com/couchbase/couchbase-operator/pkg/cluster/reconcile.go:177\ngithub.com/couchbase/couchbase-operator/pkg/cluster.(*Cluster).runReconcile\n\tgithub.com/couchbase/couchbase-operator/pkg/cluster/cluster.go:492\ngithub.com/couchbase/couchbase-operator/pkg/cluster.(*Cluster).Update\n\tgithub.com/couchbase/couchbase-operator/pkg/cluster/cluster.go:535\ngithub.com/couchbase/couchbase-operator/pkg/controller.(*CouchbaseClusterReconciler).Reconcile\n\tgithub.com/couchbase/couchbase-operator/pkg/controller/controller.go:90\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:22
The upgrade is from 7.1.5 to 7.2.4
