Uploaded image for project: 'Couchbase Kubernetes'
  1. Couchbase Kubernetes
  2. K8S-1872

2 buckets with the same spec.name cause rebalance loop

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.2, 2.0.3
    • 2.2.0
    • kubernetes
    • None
    • 4: Deliver Training, More 2.2
    • 1

    Description

      Description:

      The Operator does not prevent bucket creation where the bucket shares the same `spec.name` which then leads to a rebalance loop.

      Steps to re-create:

      1) Create 2 buckets with the following:

      apiVersion: couchbase.com/v2
      		 
      kind: CouchbaseBucket
      		 
      metadata:
      		 
        name: bucket-test
      		 
        labels:
      		 
          cluster: cb-bucket
      		 
      spec:
      		 
        name: travel-sample
      		 
        memoryQuota: 256Mi
      		 
        replicas: 2
      		 
        ioPriority: low
      		 
        evictionPolicy: valueOnly
      		 
        conflictResolution: seqno
      		 
        enableFlush: false
      		 
        enableIndexReplica: true
      		 
        compressionMode: passive
      		 
      ---


      apiVersion: couchbase.com/v2
      		 
      kind: CouchbaseBucket
      		 
      metadata:
      		 
        name: bucket-test2
      		 
        labels:
      		 
          cluster: cb-bucket
      		 
      spec:
      		 
        name: travel-sample
      		 
        memoryQuota: 256Mi
      		 
        replicas: 1
      		 
        ioPriority: low
      		 
        evictionPolicy: valueOnly
      		 
        conflictResolution: seqno
      		 
        enableFlush: false
      		 
        enableIndexReplica: true
      		 
        compressionMode: passive
      		 
      ---

      Note that because the spec.name is the same, only 1 bucket is created in the UI, but there are 2 bucket resources with different replicas numbers.

      Problem
      Because there is only 1 bucket, but the replicas config is different for the 2 bucket resources, this causes a rebalance loop

      {"level":"info","ts":1608668413.5445578,"logger":"couchbaseutil","msg":"Rebalancing","cluster":"default/cb-example","progress":0}
      		 
      {"level":"info","ts":1608668417.5856192,"logger":"couchbaseutil","msg":"Rebalancing","cluster":"default/cb-example","progress":11.17074490238426}
      		 
      {"level":"info","ts":1608668421.5996935,"logger":"couchbaseutil","msg":"Rebalancing","cluster":"default/cb-example","progress":23.50740198470799}
      		 
      {"level":"info","ts":1608668425.6037364,"logger":"couchbaseutil","msg":"Rebalancing","cluster":"default/cb-example","progress":33.33333333333334}
      		 
      {"level":"info","ts":1608668429.61321,"logger":"cluster","msg":"Rebalance completed successfully","cluster":"default/cb-example"}
      		 
      {"level":"info","ts":1608668429.657344,"logger":"cluster","msg":"Reconcile completed","cluster":"default/cb-example"}
      		 
      {"level":"info","ts":1608668429.6791978,"logger":"cluster","msg":"Resource updated","cluster":"default/cb-example","diff":"  strings.Join({\n  \t... // 7 identical lines\n  \t\"name: travel-sample\",\n  \t`password: \"\"`,\n- \t\"replicas: 2\",\n+ \t\"replicas: 1\",\n  \t\"type: couchbase\",\n  \t\"\",\n  }, \"\\n\")\n"}
      		 
      {"level":"info","ts":1608668429.684283,"logger":"cluster","msg":"Bucket updated","cluster":"default/cb-example","name":"travel-sample"}
      		 
      {"level":"info","ts":1608668430.24219,"logger":"couchbaseutil","msg":"Cluster status","cluster":"default/cb-example","balance":"unbalanced","rebalancing":false}
      		 
      {"level":"info","ts":1608668430.2422225,"logger":"couchbaseutil","msg":"Node status","cluster":"default/cb-example","name":"cb-example-0001","version":"6.5.1","class":"data_services","managed":true,"status":"active"}
      		 
      {"level":"info","ts":1608668430.2422285,"logger":"couchbaseutil","msg":"Node status","cluster":"default/cb-example","name":"cb-example-0002","version":"6.5.1","class":"data_services","managed":true,"status":"active"}
      		 
      {"level":"info","ts":1608668430.2422316,"logger":"couchbaseutil","msg":"Node status","cluster":"default/cb-example","name":"cb-example-0003","version":"6.5.1","class":"data_services","managed":true,"status":"active"}
      		 
      {"level":"info","ts":1608668434.0608795,"logger":"couchbaseutil","msg":"Rebalancing","cluster":"default/cb-example","progress":0}
      		 
      {"level":"info","ts":1608668438.0969598,"logger":"couchbaseutil","msg":"Rebalancing","cluster":"default/cb-example","progress":16.95932546071874}
      		 
      {"level":"info","ts":1608668442.107232,"logger":"couchbaseutil","msg":"Rebalancing","cluster":"default/cb-example","progress":31.41174175038058}
      		 
      {"level":"info","ts":1608668446.1163557,"logger":"cluster","msg":"Rebalance completed successfully","cluster":"default/cb-example"}
      		 
      {"level":"info","ts":1608668446.1664069,"logger":"cluster","msg":"Reconcile completed","cluster":"default/cb-example"}
      		 
      {"level":"info","ts":1608668446.1868594,"logger":"cluster","msg":"Resource updated","cluster":"default/cb-example","diff":"  strings.Join({\n  \t... // 7 identical lines\n  \t\"name: travel-sample\",\n  \t`password: \"\"`,\n- \t\"replicas: 1\",\n+ \t\"replicas: 2\",\n  \t\"type: couchbase\",\n  \t\"\",\n  }, \"\\n\")\n"}
      		 
      {"level":"info","ts":1608668446.1911297,"logger":"cluster","msg":"Bucket updated","cluster":"default/cb-example","name":"travel-sample"}
      		 
      {"level":"info","ts":1608668447.1387665,"logger":"couchbaseutil","msg":"Cluster status","cluster":"default/cb-example","balance":"unbalanced","rebalancing":false}

      We should not allow 2 buckets with the same `spec.name` to be created even though the `metadata.name` is different

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          Ascending order - Click to sort in descending order
          tin.tran Tin Tran created issue -
          tin.tran Tin Tran made changes -
          Field Original Value New Value
          Affects Version/s 2.0.2 [ 16799 ]
          Affects Version/s 2.0.3 [ 17217 ]
          tin.tran Tin Tran made changes -
          Summary 2 buckets with the same spec.name creates rebalance loop 2 buckets with the same spec.name causes rebalance loop
          tin.tran Tin Tran made changes -
          Summary 2 buckets with the same spec.name causes rebalance loop 2 buckets with the same spec.name cause rebalance loop
          simon.murray Simon Murray made changes -
          Rank Ranked lower
          simon.murray Simon Murray made changes -
          Rank Ranked higher
          simon.murray Simon Murray made changes -
          Rank Ranked higher
          simon.murray Simon Murray made changes -
          Fix Version/s 2.2.0 [ 17035 ]
          simon.murray Simon Murray made changes -
          Rank Ranked higher
          simon.murray Simon Murray made changes -
          Sprint 4: Deliver Training, More 2.2 [ 1344 ]
          simon.murray Simon Murray made changes -
          Rank Ranked higher
          simon.murray Simon Murray made changes -
          Resolution Fixed [ 1 ]
          Status Open [ 1 ] Closed [ 6 ]

          People

            simon.murray Simon Murray
            tin.tran Tin Tran
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty