Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-964

K8S Helm Chart not validating on EKS

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 1.2.0
    • 1.2.1
    • helm

    Description

      REQ

      I was trying to test the HELM chart deployment and when I run the following command:

      helm install couchbase/couchbase-operator

      I get the following error:

      Error: validation failed: error validating "": error validating data: [ValidationError(Deployment.spec.template.spec.containers[0]): unknown field "failureThreshold" in io.k8s.api.core.v1.Container, ValidationError(Deployment.spec.template.spec.containers[0]): unknown field "httpGet" in io.k8s.api.core.v1.Container, ValidationError(Deployment.spec.template.spec.containers[0]): unknown field "initialDelaySeconds" in io.k8s.api.core.v1.Container, ValidationError(Deployment.spec.template.spec.containers[0]): unknown field "periodSeconds" in io.k8s.api.core.v1.Container, ValidationError(Deployment.spec.template.spec.containers[0].resources): unknown field "cpu" in io.k8s.api.core.v1.ResourceRequirements, ValidationError(Deployment.spec.template.spec.containers[0].resources): unknown field "memory" in io.k8s.api.core.v1.ResourceRequirements]
      

      BACKGROUND

      I am attempting to deploy to an EKS cluster running kubernetes 1.12 in the US-EAST-1 Region.

      I installed HELM and TIller successfully using the RBAC-Tiller.yaml specified in our documentation.

      HQ-Computer:helm craigkovar$ helm version
      Client: &version.Version{SemVer:"v2.14.0", GitCommit:"05811b84a3f93603dd6c2fcfe57944dfa7ab7fd0", GitTreeState:"clean"}
      Server: &version.Version{SemVer:"v2.14.0", GitCommit:"05811b84a3f93603dd6c2fcfe57944dfa7ab7fd0", GitTreeState:"clean"}

      Also

       

      HQ-Computer:helm craigkovar$ helm search couchbase
      NAME CHART VERSION APP VERSION DESCRIPTION
      couchbase/couchbase-cluster 0.1.2 1.2 Couchbase Server is a NoSQL document database with a dist...
      couchbase/couchbase-operator 0.1.2 1.2 A Helm chart to deploy the Couchbase Autonomous Operator ...
      HQ-Computer:helm craigkovar${noformat}
       

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            simon.murray Simon Murray added a comment -

            Morning Craig, I've put a potential fix into the couchbase-partners repos.  Mind checking if it's working any better now?  I've not bumped the revision yet so you'll need to uninstall and reinstall the chart.

            simon.murray Simon Murray added a comment - Morning Craig, I've put a potential fix into the couchbase-partners repos.  Mind checking if it's working any better now?  I've not bumped the revision yet so you'll need to uninstall and reinstall the chart.

            Hey Simon Murray,  I switched my cluster over to OpenShift 4.0 for another demo,  but the below command worked

            helm install couchbase/couchbase-operator

            and we can see the correct resources created.  When I try and deploy the cluster through helm I am getting a Context Deadline Exceeded as shown by this error from the operator log but no errors thrown by helm

            error msg="Cluster setup failed: context deadline exceeded: Pod Creation error - error creating pod cbdemo-couchbase-cluster-0000" cluster-name=cbdemo-couchbase-cluster module=cluster

            but was able to deploy by hand.

            cb-example-0000                                                   1/1     Running   0          8m22s
            cb-example-0001                                                   1/1     Running   0          5m59s
            cb-example-0002                                                   1/1     Running   0          3m39s
            tailored-umbrellabird-couchbase-admission-controller-75d56gk5x6   1/1     Running   0          23m
            tailored-umbrellabird-couchbase-operator-57c45fcff8-jll6g         1/1     Running   0          23m

            I will continue to look into this.

            craig.kovar Craig Kovar (Inactive) added a comment - Hey Simon Murray ,  I switched my cluster over to OpenShift 4.0 for another demo,  but the below command worked helm install couchbase/couchbase-operator and we can see the correct resources created.  When I try and deploy the cluster through helm I am getting a Context Deadline Exceeded as shown by this error from the operator log but no errors thrown by helm error msg="Cluster setup failed: context deadline exceeded: Pod Creation error - error creating pod cbdemo-couchbase-cluster-0000" cluster-name=cbdemo-couchbase-cluster module=cluster but was able to deploy by hand. cb-example-0000 1/1 Running 0 8m22s cb-example-0001 1/1 Running 0 5m59s cb-example-0002 1/1 Running 0 3m39s tailored-umbrellabird-couchbase-admission-controller-75d56gk5x6 1/1 Running 0 23m tailored-umbrellabird-couchbase-operator-57c45fcff8-jll6g 1/1 Running 0 23m I will continue to look into this.
            simon.murray Simon Murray added a comment -

            I'd not bother, RHEL is bloated and takes an age to download.  Our timeouts default to 5 minutes which is your issue.  You can override the operator CLI parameters and set it to something bigger e.g. 15m.  See https://docs.couchbase.com/operator/1.2/operator-config.html#command-line-arguments

            simon.murray Simon Murray added a comment - I'd not bother, RHEL is bloated and takes an age to download.  Our timeouts default to 5 minutes which is your issue.  You can override the operator CLI parameters and set it to something bigger e.g. 15m.  See https://docs.couchbase.com/operator/1.2/operator-config.html#command-line-arguments
            ingenthr Matt Ingenthron added a comment - - edited

            This wasn't part of GA directly, but was already pushed to helm. Simon Murray is going to look at a couple more things here.

            • Perms
            • Version

            After this Craig Kovar, we'd like you to verify further. Look out for the ping from Simon Murray.

            ingenthr Matt Ingenthron added a comment - - edited This wasn't part of GA directly, but was already pushed to helm. Simon Murray is going to look at a couple more things here. Perms Version After this Craig Kovar , we'd like you to verify further. Look out for the ping from Simon Murray .

            Matt Ingenthron sure thing.  Just let me know and I can do some validation.

            craig.kovar Craig Kovar (Inactive) added a comment - Matt Ingenthron sure thing.  Just let me know and I can do some validation.
            simon.murray Simon Murray added a comment -

            Follow ups in K8S-967

            simon.murray Simon Murray added a comment - Follow ups in K8S-967

            Craig Kovar: Updates are pushed to the repo, can you help validate?

             

            https://github.com/couchbase-partners/helm-charts

            ingenthr Matt Ingenthron added a comment - Craig Kovar : Updates are pushed to the repo, can you help validate?   https://github.com/couchbase-partners/helm-charts

            Matt Ingenthron and Simon Murray

            Hey,  I was testing this and did notice an issue with deploying on OpenShift 4.1.  Here are the steps I tested in EKS and OpenShift

            EKS

            helm install --values myvalues.yaml couchbase/couchbase-operator --namespace cbtesting --name cbtesting

            Passed  (myvalues.yaml as below)

            couchbaseOperator:
              commandArgs:
                create-crd: true
                pod-create-timeout: 10m

            Then I ran

            helm install --set couchbaseCluster.version=enterprise-5.5.3 --name cbcluster couchbase/couchbase-cluster --namespace cbtesting

            Passed.   I also tried downgrading which the Admission Controller correctly caught as that is not supported.

             

            OpenShift

            oc new-project cbtesting
            helm install --values myvalues.yaml couchbase/couchbase-operator --namespace cbtesting --name cbtesting

            Passed

            However,  when I now try to run

            helm install --name cbcluster couchbase/couchbase-cluster --namespace cbtesting

            Fail with the following error:

            Error: validation failed: error validating "": error validating data: ValidationError(CouchbaseCluster.spec): unknown field "securityContext" in com.couchbase.v1.CouchbaseCluster.spec

            I know OpenShift 4.1 is still in DP so this may not be a valid test

             

            craig.kovar Craig Kovar (Inactive) added a comment - Matt Ingenthron and Simon Murray Hey,  I was testing this and did notice an issue with deploying on OpenShift 4.1.  Here are the steps I tested in EKS and OpenShift EKS helm install --values myvalues.yaml couchbase/couchbase-operator --namespace cbtesting --name cbtesting Passed  (myvalues.yaml as below) couchbaseOperator: commandArgs: create-crd: true pod-create-timeout: 10m Then I ran helm install --set couchbaseCluster.version=enterprise-5.5.3 --name cbcluster couchbase/couchbase-cluster --namespace cbtesting Passed.   I also tried downgrading which the Admission Controller correctly caught as that is not supported.   OpenShift oc new-project cbtesting helm install --values myvalues.yaml couchbase/couchbase-operator --namespace cbtesting --name cbtesting Passed However,  when I now try to run helm install --name cbcluster couchbase/couchbase-cluster --namespace cbtesting Fail with the following error: Error: validation failed: error validating "": error validating data: ValidationError(CouchbaseCluster.spec): unknown field "securityContext" in com.couchbase.v1.CouchbaseCluster.spec I know OpenShift 4.1 is still in DP so this may not be a valid test  
            simon.murray Simon Murray added a comment -

            So while that field is correctly named, in the right place in the spec it's not in the CRD schema.  I'm assuming that helm is now validating what's in the chart matches what's advertised.  While technically this is ostensibly a packaging change to add a new CRD, it's also a point release, so we cannot fix until 1.2.1.

            simon.murray Simon Murray added a comment - So while that field is correctly named, in the right place in the spec it's not in the CRD schema.  I'm assuming that helm is now validating what's in the chart matches what's advertised.  While technically this is ostensibly a packaging change to add a new CRD, it's also a point release, so we cannot fix until 1.2.1.
            simon.murray Simon Murray added a comment -

            See comments

            simon.murray Simon Murray added a comment - See comments
            simon.murray Simon Murray added a comment -

            After some digging the change that caused the error was fixed in 2.14.1, so please upgrade helm/tiller and it will go away

            simon.murray Simon Murray added a comment - After some digging the change that caused the error was fixed in 2.14.1, so please upgrade helm/tiller and it will go away
            simon.murray Simon Murray added a comment -

            As stated previously this is a bug in Helm that has been reverted, so closing... for now

            simon.murray Simon Murray added a comment - As stated previously this is a bug in Helm that has been reverted, so closing... for now

            Summary: Due to a known issue in Helm 2.14.0, attempting to install the Couchbase cluster would result in validation errors, even for correctly-configured charts.

            Workaround: Use a version of Helm other than 2.14.0.

            eric.schneider Eric Schneider (Inactive) added a comment - Summary : Due to a known issue in Helm 2.14.0, attempting to install the Couchbase cluster would result in validation errors, even for correctly-configured charts. Workaround : Use a version of Helm other than 2.14.0.

            People

              simon.murray Simon Murray
              craig.kovar Craig Kovar (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty