Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-2205

[OCP] Cannot Provision secret to deploy Couchbase Cluster on OCP v4.4

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 2.2.0
    • operator
    • None
    • 1

    Description

      Job: http://qa.sc.couchbase.com/view/Cloud/job/k8s-cbop-oc-pipeline/109/console

      OCP: 4.4 (based on Kubernetes version 1.17)

      Server: registry.connect.redhat.com/couchbase/server:6.6.2-1

      Operator: registry.gitlab.com/cb-rhcc/operator:latest {build 228}

      Test: TestCreateCluster

      BackTrace:

      03:33:42     crd_util.go:51: creating couchbase cluster: test-couchbase-h68lf
      03:48:49     util.go:1288: timeout: size does not match, wanted 3, got 0
      03:48:49     util.go:1289: goroutine 1169 [running]:
      03:48:49         runtime/debug.Stack(0x1f09834, 0xc00095cc98, 0xc00095cc90)
      03:48:49         	/jenkins/workspace/k8s-cbop-oc-pipeline/go/src/runtime/debug/stack.go:24 +0xab
      03:48:49         github.com/couchbase/couchbase-operator/test/e2e/e2eutil.Die(0xc00084ac00, 0x3525400, 0xc00078e3e0)
      03:48:49         	/jenkins/workspace/k8s-cbop-oc-pipeline/test/e2e/e2eutil/util.go:1284 +0x34
      03:48:49         github.com/couchbase/couchbase-operator/test/e2e/e2eutil.MustWaitClusterStatusHealthy(0xc00084ac00, 0xc00011e8f0, 0xc0005ee000, 0xd18c2e2800)
      03:48:49         	/jenkins/workspace/k8s-cbop-oc-pipeline/test/e2e/e2eutil/wait_util.go:580 +0x94
      03:48:49         github.com/couchbase/couchbase-operator/test/e2e/e2eutil.MustNewClusterFromSpec(0xc00084ac00, 0xc00011e8f0, 0xc00040d400, 0x0)
      03:48:49         	/jenkins/workspace/k8s-cbop-oc-pipeline/test/e2e/e2eutil/util.go:88 +0x9d
      03:48:49         github.com/couchbase/couchbase-operator/test/e2e/e2eutil.(*ClusterOptions).MustCreate(0xc000b67e78, 0xc00084ac00, 0xc00011e8f0, 0xc0009aa140)
      03:48:49         	/jenkins/workspace/k8s-cbop-oc-pipeline/test/e2e/e2eutil/util.go:422 +0x5d
      03:48:49         github.com/couchbase/couchbase-operator/test/e2e.TestCreateCluster(0xc00084ac00)
      03:48:49         	/jenkins/workspace/k8s-cbop-oc-pipeline/test/e2e/basic_test.go:27 +0x705
      03:48:49         testing.tRunner(0xc00084ac00, 0x2b1c140)
      03:48:49         	/jenkins/workspace/k8s-cbop-oc-pipeline/go/src/testing/testing.go:1193 +0x203
      03:48:49         created by testing.(*T).Run
      03:48:49         	/jenkins/workspace/k8s-cbop-oc-pipeline/go/src/testing/testing.go:1238 +0x5d8
      03:48:49         
      03:48:49 time="2021-05-21T03:48:41-07:00" level=info msg="TestOperator/TestCreateCluster ✗"
      03:48:49 --- FAIL: TestOperator (0.00s)
      03:48:49     --- FAIL: TestOperator/TestCreateCluster (914.45s) 

      Error:

      "msg":"Reconciler error","controller":"couchbase-controller","name":"test-couchbase-h68lf","namespace":"test-g79ds","error":"secrets \"test-couchbase-h68lf\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>"

      (cbopinfo attached)

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            The error is related to RBAC and I'll re-check if this is a env. config issue.

            prateek.kumar Prateek Kumar (Inactive) added a comment - The error is related to RBAC and I'll re-check if this is a env. config issue.
            simon.murray Simon Murray added a comment -

            Try the attached review, see what happens, while I'm making sure this doesn't break the thousands of other distros.

            simon.murray Simon Murray added a comment - Try the attached review, see what happens, while I'm making sure this doesn't break the thousands of other distros.
            simon.murray Simon Murray added a comment -

            In other news.... my magical new condition has made debug super easy!

            simon.murray Simon Murray added a comment - In other news.... my magical new condition has made debug super easy!

            Woahh, this is extremely nice!, You'll have to teach this sorcery to us mortals as well.

            prateek.kumar Prateek Kumar (Inactive) added a comment - Woahh, this is extremely nice!, You'll have to teach this sorcery to us mortals as well.
            simon.murray Simon Murray added a comment - No secret... https://github.com/spjmurray/logify/

            The fix given in the review works and the secret is created successfully. However we get another error while provisioning Cluster pods.

            Error:

            "msg":"Reconciliation failed","cluster":"test-kmgqx/test-couchbase-6xd85","error":"fail to create member's pod (test-couchbase-6xd85-0000): pods \"test-couchbase-6xd85-0000\" is forbidden: unable to validate against any security context constraint: [spec.containers[0].securityContext.securityContext.runAsUser: Invalid value: 1000: must be in the ranges: [1000590000, 1000599999]]"

            prateek.kumar Prateek Kumar (Inactive) added a comment - The fix given in the review works and the secret is created successfully. However we get another error while provisioning Cluster pods. Error: "msg" : "Reconciliation failed" , "cluster" : "test-kmgqx/test-couchbase-6xd85" , "error" : "fail to create member's pod (test-couchbase-6xd85-0000): pods \"test-couchbase-6xd85-0000\" is forbidden: unable to validate against any security context constraint: [spec.containers[0].securityContext.securityContext.runAsUser: Invalid value: 1000: must be in the ranges: [1000590000, 1000599999]]"
            simon.murray Simon Murray added a comment -

            FFFFFFFUUUUUUUUUUUUUU! This is a testware issue, in "test/e2e/e2eutil/crd_util.go" (I think) there is a bit that adds "runAsUser: 1000", remove that on OCP.... I think the platform type is propagated up to that function now finger crossed

            simon.murray Simon Murray added a comment - FFFFFFFUUUUUUUUUUUUUU! This is a testware issue, in "test/e2e/e2eutil/crd_util.go" (I think) there is a bit that adds "runAsUser: 1000", remove that on OCP.... I think the platform type is propagated up to that function now finger crossed

            removed runAsUser while running with OCP: 

            === RUN   TestOperator
            === RUN   TestOperator/TestCreateCluster
            === PAUSE TestOperator/TestCreateCluster
            === CONT  TestOperator/TestCreateCluster
            time="2021-05-21T19:05:19+05:30" level=info msg="TestOperator/TestCreateCluster ✔"
            --- PASS: TestOperator (0.00s)
                --- PASS: TestOperator/TestCreateCluster (258.35s)
                    crd_util.go:54: creating couchbase cluster: test-couchbase-ccrtx
            PASS 

            prateek.kumar Prateek Kumar (Inactive) added a comment - removed runAsUser while running with OCP:  === RUN   TestOperator === RUN   TestOperator/TestCreateCluster === PAUSE TestOperator/TestCreateCluster === CONT  TestOperator/TestCreateCluster time= "2021-05-21T19:05:19+05:30" level=info msg= "TestOperator/TestCreateCluster ✔" --- PASS: TestOperator ( 0 .00s)     --- PASS: TestOperator/TestCreateCluster ( 258 .35s)         crd_util.go: 54 : creating couchbase cluster: test-couchbase-ccrtx PASS

            The fix given resolves the issue and we no longer see the same on Openshift v4.4

            prateek.kumar Prateek Kumar (Inactive) added a comment - The fix given resolves the issue and we no longer see the same on Openshift v4.4

            People

              simon.murray Simon Murray
              prateek.kumar Prateek Kumar (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty