Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-3390

TestBucketAddRemoveBasic is failing on Server 7.6

    XMLWordPrintable

Details

    • Bug
    • Resolution: Not a Bug
    • Blocker
    • 2.7.0
    • None
    • None
    • None
    • 5 - Battle to the End
    • 1

    Description

      On RC7 of Server 7.6 this test is failing

      http://qa.sc.couchbase.com/view/Cloud/job/k8s-cbop-gke-pipeline-26x/82/testReport/(root)/custom/TestBucketAddRemoveBasic/

      >  goroutine 374 [running]:
      01:31:28 	>  runtime/debug.Stack()
      01:31:28 	>  	runtime/debug/stack.go:24 +0x72
      01:31:28 	>  sigs.k8s.io/controller-runtime/pkg/log.eventuallyFulfillRoot()
      01:31:28 	>  	sigs.k8s.io/controller-runtime@v0.16.3/pkg/log/log.go:60 +0xf2
      01:31:28 	>  sigs.k8s.io/controller-runtime/pkg/log.(*delegatingLogSink).Enabled(0xc0004f8000, 0xc000a86f38?)
      01:31:28 	>  	sigs.k8s.io/controller-runtime@v0.16.3/pkg/log/deleg.go:111 +0x3c
      01:31:28 	>  github.com/go-logr/logr.Logger.Enabled({{0x6c97010?, 0xc0004f8000?}, 0x1414f59?})
      01:31:28 	>  	github.com/go-logr/logr@v1.3.0/logr.go:267 +0x59
      01:31:28 	>  github.com/couchbase/couchbase-operator/pkg/util/couchbaseutil.Client.doRequest({{0xc000cef923, 0xd}, {0xc000cef938, 0x8}, 0x0, {0x6c7ecf0, 0xc000948c00}, {0x6c91d10, 0xc0001bc008}, {0x0, ...}}, ...)
      01:31:28 	>  	github.com/couchbase/couchbase-operator/pkg/util/couchbaseutil/core.go:156 +0x365
      01:31:28 	>  github.com/couchbase/couchbase-operator/pkg/util/couchbaseutil.(*Client).Get(0xc00010e420, 0xc0007ce770, {0xc000c0d3e0, 0x2f})
      01:31:28 	>  	github.com/couchbase/couchbase-operator/pkg/util/couchbaseutil/core.go:261 +0x20a
      01:31:28 	>  github.com/couchbase/couchbase-operator/pkg/util/couchbaseutil.(*Request).On.func1(0xc000a87478?, 0xc0007ce770)
      01:31:28 	>  	github.com/couchbase/couchbase-operator/pkg/util/couchbaseutil/api.go:222 +0xf9
      01:31:28 	>  github.com/couchbase/couchbase-operator/pkg/util/couchbaseutil.(*Request).On(0xc0007ce770, 0x2f?, {0x4f6dbc0?, 0xc000800320?})
      01:31:28 	>  	github.com/couchbase/couchbase-operator/pkg/util/couchbaseutil/api.go:249 +0x55e
      01:31:28 	>  github.com/couchbase/couchbase-operator/test/e2e/e2eutil.WaitUntilBucketsExist.func2()
      01:31:28 	>  	github.com/couchbase/couchbase-operator/test/e2e/e2eutil/wait_util.go:469 +0x273
      01:31:28 	>  github.com/couchbase/couchbase-operator/pkg/util/retryutil.Retry({0x6c91d48, 0xc0009dc960}, 0x1419797?, 0xc000a87960)
      01:31:28 	>  	github.com/couchbase/couchbase-operator/pkg/util/retryutil/retryutil.go:14 +0xb6
      01:31:28 	>  github.com/couchbase/couchbase-operator/test/e2e/e2eutil.WaitUntilBucketsExist(0xc00020c700, 0xc0007f2b00, {0xc000a87a10, 0x1, 0x1}, 0x14197c5?)
      01:31:28 	>  	github.com/couchbase/couchbase-operator/test/e2e/e2eutil/wait_util.go:482 +0x209
      01:31:28 	>  github.com/couchbase/couchbase-operator/test/e2e/e2eutil.MustWaitUntilBucketExists(0xc001047e18?, 0xc00020c700?, 0x6cb10f8?, {0x5571840, 0xc0007fa1c0}, 0x1414f59?)
      

      Pods appear to get into an odd "Warming up state" after 2 of the 3 buckets are created, preventing reconciliation from adding the third bucket

      {"level":"info","ts":"2024-02-28T08:54:09Z","logger":"cluster","msg":"Aborting topology reconcile","cluster":"test-4psb8/test-couchbase-zp9pv","reason":"pods are warming up"}
      

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              gilad.kalchheim Gilad Kalchheim
              justin.ashworth Justin Ashworth
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty