Service Catalog is unable to provision CND Operator

Below are the steps used to provision CND Operator using Service Broker:

1. Install Service Catalog using helm:

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ helm repo add svc-cat https://kubernetes-sigs.github.io/service-catalog

   "svc-cat" has been added to your repositories

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ helm install catalog svc-cat/catalog

   NAME: catalog

   LAST DEPLOYED: Mon Mar 22 21:17:06 2021

   NAMESPACE: default

   STATUS: deployed

   REVISION: 1

   TEST SUITE: None

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ curl -sLO https://download.svcat.sh/cli/latest/darwin/amd64/svcat

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ chmod +x ./svcat

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ mv ./svcat /usr/local/bin/

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ svcat version --client

   Client Version: v0.3.1

2. Install 2.1.0 CRD:

   Prateeks-MacBook-Pro:couchbase-autonomous-operator-kubernetes_2.1.0-macos-x86_64 prateekkumar$ kubectl create -f crd.yaml

   customresourcedefinition.apiextensions.k8s.io/couchbaseautoscalers.couchbase.com created

   customresourcedefinition.apiextensions.k8s.io/couchbasebackuprestores.couchbase.com created

   customresourcedefinition.apiextensions.k8s.io/couchbasebackups.couchbase.com created

   customresourcedefinition.apiextensions.k8s.io/couchbasebuckets.couchbase.com created

   customresourcedefinition.apiextensions.k8s.io/couchbaseclusters.couchbase.com created

   customresourcedefinition.apiextensions.k8s.io/couchbaseephemeralbuckets.couchbase.com created

   customresourcedefinition.apiextensions.k8s.io/couchbasegroups.couchbase.com created

   customresourcedefinition.apiextensions.k8s.io/couchbasememcachedbuckets.couchbase.com created

   customresourcedefinition.apiextensions.k8s.io/couchbasereplications.couchbase.com created

   customresourcedefinition.apiextensions.k8s.io/couchbaserolebindings.couchbase.com created

   customresourcedefinition.apiextensions.k8s.io/couchbaseusers.couchbase.com created

3. Status of the pods:

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ kubectl get pods

   NAME                                                  READY   STATUS    RESTARTS   AGE

   catalog-catalog-controller-manager-65f6567d8c-4x4t7   1/1     Running   0          8m5s

   catalog-catalog-webhook-d6694bdf8-zk4pq               1/1     Running   0          8m5s

   couchbase-operator-admission-865945c6cd-2jvmz         1/1     Running   0          71s

4. Install Service Broker CRD:

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ kubectl create -f crd.yaml

   customresourcedefinition.apiextensions.k8s.io/servicebrokerconfigs.servicebroker.couchbase.com created

5. Install Cluster Service Broker:

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ ./cbsbctl.darwin create clusterservicebroker --profile=demo

   clusterroles/couchbase-service-broker created

   servicebrokerconfigs/couchbase-service-broker created

   serviceaccounts/couchbase-service-broker created

   clusterrolebindings/couchbase-service-broker created

   secrets/couchbase-service-broker created

   deployments/couchbase-service-broker created

   services/couchbase-service-broker created

   clusterservicebrokers/couchbase-service-broker created

6. Status of the pods:

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ kubectl get pods

   NAME                                                  READY   STATUS    RESTARTS   AGE

   catalog-catalog-controller-manager-65f6567d8c-4x4t7   1/1     Running   0          14m

   catalog-catalog-webhook-d6694bdf8-zk4pq               1/1     Running   0          14m

   couchbase-operator-admission-865945c6cd-2jvmz         1/1     Running   0          7m50s

   couchbase-service-broker-778556c74c-mdhdx             1/1     Running   0          82s

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ kubectl get clusterservicebrokers

   NAME                       URL                                        STATUS   AGE

   couchbase-service-broker   https://couchbase-service-broker.default   Ready    56s

7. Classes available:

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ svcat get class

   NAME            NAMESPACE            BROKER                     DESCRIPTION

   +-----------------------+-----------+--------------------------+--------------------------------+

   couchbase-operator                  couchbase-service-broker   Couchbase Autunomous Operator

   couchbase-osb-service               couchbase-service-broker   Couchbase Server instance with

   a single shared bucket

8. Plans available:

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ svcat get plans

   NAME       NAMESPACE           CLASS                    DESCRIPTION

   +--------------+-----------+-----------------------+--------------------------------+

   2.1.0                      couchbase-operator      Couchbase Autonomous Operator

   2.1.0

   csb-standard               couchbase-osb-service   Basic Couchbase Server

   cluster with uni-directional

   replication

   csb-basic                  couchbase-osb-service   Basic Couchbase Server cluster

9. Provision 2.1.0 Operator:

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ svcat provision cao --plan 2.1.0 --class=couchbase-operator password=operator

   Name:        cao

   Namespace:   default

   Status:

   Class:       056611fa-1cf5-488a-b080-010f19bf099b

   Plan:        edd5c808-4460-470f-afad-1b6638cec614

   Parameters:

   No parameters defined

10. The provisioning of the operator fails:

    Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ kubectl get serviceinstances

    NAME   CLASS                                                      PLAN                                   STATUS   AGE

    cao    ClusterServiceClass/056611fa-1cf5-488a-b080-010f19bf099b   edd5c808-4460-470f-afad-1b6638cec614   Failed   21s

    Status:

    Status:

    Async Op In Progress:  false

    Conditions:

    Last Transition Time:         2021-03-23T08:45:53Z

    Message:                      Orphan mitigation was completed successfully

    Reason:                       OrphanMitigationSuccessful

    Status:                       False

    Type:                         Ready

    Last Transition Time:         2021-03-23T08:45:54Z

    Message:                      Provision call failed: the server could not find the requested resource

    Reason:                       ProvisionCallFailed

    Status:                       True

    Type:                         Failed

    Deprovision Status:             Succeeded

    Last Condition State:           Failed

    Observed Generation:            2

    Orphan Mitigation In Progress:  false

    Provision Status:               NotProvisioned

    Reconciled Generation:          0

    User Specified Class Name:      ClusterServiceClass/056611fa-1cf5-488a-b080-010f19bf099b

    User Specified Plan Name:       edd5c808-4460-470f-afad-1b6638cec614

    Events:

    Type     Reason                            Age                From                                Message

    ----     ------                            ----               ----                                -------

    Normal   Provisioning                      60s                service-catalog-controller-manager  The instance is being provisioned asynchronously

    Warning  ProvisionCallFailed               59s (x2 over 59s)  service-catalog-controller-manager  Provision call failed: the server could not find the requested resource

    Warning  StartingInstanceOrphanMitigation  59s                service-catalog-controller-manager  The instance provision call failed with an ambiguous error; attempting to deprovision the instance in order to mitigate an orphaned resource

    Normal   Deprovisioning                    59s                service-catalog-controller-manager  The instance is being deprovisioned asynchronously

    Normal   OrphanMitigationSuccessful        57s                service-catalog-controller-manager  Orphan mitigation was completed successfully

    Logs attached (SB_Log.txt)
    **

1. In Step 9 I provision Operator and then had plans to provision Couchbase Cluster , but the operator provisioning failed.
2. Another way might be to provision Cluster itself and it'll take care of the operator provisioning.
3. Step 1-5 was executed on a fresh cluster.
4. We get the same output for classes and plans again as mentioned in Step 7-8.
5. Provision Basic Couchbase Server cluster:

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ svcat provision demo --class=couchbase-osb-service --plan=csb-basic password=operator

   Name:        demo

   Namespace:   default

   Status:

   Class:       da21b71c-2c10-4a57-8773-fde45bb339ff

   Plan:        ec0f2c9b-0277-46d7-985f-ba1fbf3b068d

   Parameters:

   No parameters defined

   Status:

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ svcat get instances

   NAME   NAMESPACE                  CLASS                                   PLAN                   STATUS

   +------+-----------+--------------------------------------+--------------------------------------+--------+

   demo   default     da21b71c-2c10-4a57-8773-fde45bb339ff   ec0f2c9b-0277-46d7-985f-ba1fbf3b068d   Failed

   Prateeks-MacBook-Pro:couchbase-service-broker-kubernetes prateekkumar$ kubectl get serviceinstances

   NAME   CLASS                                                      PLAN                                   STATUS   AGE

   demo   ClusterServiceClass/da21b71c-2c10-4a57-8773-fde45bb339ff   ec0f2c9b-0277-46d7-985f-ba1fbf3b068d   Failed   6m39s

   Logs attached. (SB_Log_1.txt)