Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-3400

Tests failing with clusters that appear not to start, 2.7.0

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • 2.7.0
    • None
    • None
    • None
    • 5 - Forces of Destiny, 7 -The Rebellion Begins
    • 2

    Description

      There are several tests that are failing for different reasons, but the logs appear quite empty, as if the cluster did not start. The tests are:

      TestBasicMDSScaling

      TestRemoveLastDataService

      TestRemoveServerClassWithNodeService

      TestCollectionWithAnnotations

       

       

      An example for a failure:

      custom.TestBasicMDSScalingFailing for the past 1 build (Since #983 )Took 5 min 31 sec.add descriptionError Messagetimeout: size does not match, wanted 3, got 4Stacktracegoroutine 179 [running]:
      runtime/debug.Stack()
      	runtime/debug/stack.go:24 +0x72
      github.com/couchbase/couchbase-operator/test/e2e/e2eutil.Die(0xc000b37860, {0x6c85540, 0xc001a9e020})
      	github.com/couchbase/couchbase-operator/test/e2e/e2eutil/util.go:1455 +0x36
      github.com/couchbase/couchbase-operator/test/e2e/e2eutil.MustWaitClusterStatusHealthy(0xc000b37860?, 0xc000ed1b80?, 0x1?, 0x55e907a?)
      	github.com/couchbase/couchbase-operator/test/e2e/e2eutil/wait_util.go:624 +0x5b
      github.com/couchbase/couchbase-operator/test/e2e.TestBasicMDSScaling(0xc000b37860)
      	github.com/couchbase/couchbase-operator/test/e2e/cluster_test.go:603 +0x13c6
      testing.tRunner(0xc000b37860, 0x57572d0)
      	testing/testing.go:1576 +0x217
      created by testing.(*T).Run
      	testing/testing.go:1629 +0x806 

       

       

       

      The logs:

       

       

      {"level":"info","ts":"2024-03-13T11:04:22Z","logger":"main","msg":"couchbase-operator","version":"2.7.0 (build 153)","revision":"ab877cae058771a995735cf0c03b90eaf75cdd3f"}
      {"level":"debug","ts":"2024-03-13T11:04:22Z","logger":"main","msg":"Initializing resource manager."}
      {"level":"debug","ts":"2024-03-13T11:04:22Z","logger":"main","msg":"Initializing controller."}
      {"level":"debug","ts":"2024-03-13T11:04:22Z","logger":"main","msg":"Starting resource manager."}
      {"level":"info","ts":"2024-03-13T11:04:22Z","logger":"controller-runtime.metrics","msg":"Starting metrics server"}
      {"level":"info","ts":"2024-03-13T11:04:22Z","logger":"controller-runtime.metrics","msg":"Serving metrics server","bindAddress":"0.0.0.0:8383","secure":false}
      {"level":"info","ts":"2024-03-13T11:04:22Z","msg":"attempting to acquire leader lease test-z4pml/couchbase-operator..."}
      {"level":"info","ts":"2024-03-13T11:04:40Z","msg":"successfully acquired lease test-z4pml/couchbase-operator"}
      {"level":"debug","ts":"2024-03-13T11:04:40Z","logger":"events","msg":"couchbase-operator-594bbb4d8-2945c_3ebbdfc1-c46f-4649-8ed3-31f81d026510 became leader","type":"Normal","object":{"kind":"Lease","namespace":"test-z4pml","name":"couchbase-operator","uid":"1819e441-ebaf-496d-b536-7c6d20718af9","apiVersion":"coordination.k8s.io/v1","resourceVersion":"24428"},"reason":"LeaderElection"}
      {"level":"info","ts":"2024-03-13T11:04:40Z","msg":"Starting EventSource","controller":"couchbase-controller","source":"kind source: *v2.CouchbaseCluster"}
      {"level":"info","ts":"2024-03-13T11:04:40Z","msg":"Starting Controller","controller":"couchbase-controller"}
      {"level":"info","ts":"2024-03-13T11:04:40Z","msg":"Starting workers","controller":"couchbase-controller","worker count":4}
      {"level":"info","ts":"2024-03-13T11:04:40Z","logger":"cluster","msg":"Watching new cluster","cluster":"test-z4pml/test-couchbase-mv2xh"}
      {"level":"info","ts":"2024-03-13T11:05:03Z","logger":"cluster","msg":"Couchbase client starting","cluster":"test-z4pml/test-couchbase-mv2xh"}
      {"level":"info","ts":"2024-03-13T11:05:03Z","logger":"cluster","msg":"Janitor starting","cluster":"test-z4pml/test-couchbase-mv2xh"} 

       

       

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            usamah.jassat Usamah Jassat
            gilad.kalchheim Gilad Kalchheim
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty