Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-36319

[Volume Test]: Rebalance In failed during durable doc-ops while there are ~90M durable docs already present in bucket which is at 90% DGM

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Critical
    • 6.5.0
    • 6.5.0
    • couchbase-bucket
    • 6.5.0-3480

    Description

      1. Initialize a 6 nodes cluster
      2. Create a bucket with replica=1. Load data until it goes into 90% DGM.
      Bucket statistics

      +----------------+---------+----------+-----+----------+-------------+-------------+-------------+
      		 
      | Bucket         | Type    | Replicas | TTL | Items    | RAM Quota   | RAM Used    | Disk Used   |
      		 
      +----------------+---------+----------+-----+----------+-------------+-------------+-------------+
      		 
      | GleamBookUsers | membase | 1        | 0   | 91239994 | 31457280000 | 25109852744 | 66034403648 |
      		 
      +----------------+---------+----------+-----+----------+-------------+-------------+-------------+

      3. Upsert initial 30% docs and rebalance-in 1 node in parallel.

      +----------------+----------------------------+--------------+
      		 
      | Nodes          | Services                   | Status       |
      		 
      +----------------+----------------------------+--------------+
      		 
      | 172.23.106.137 | [u'kv']                    | Cluster node |
      		 
      | 172.23.106.134 | [u'index', u'kv', u'n1ql'] | Cluster node |
      		 
      | 172.23.105.168 | [u'kv']                    | Cluster node |
      		 
      | 172.23.106.82  | [u'kv']                    | Cluster node |
      		 
      | 172.23.106.136 | [u'kv']                    | Cluster node |
      		 
      | 172.23.106.138 | [u'kv']                    | Cluster node |
      		 
      | 172.23.106.83  | None                       | <--- IN ---  |
      		 
      +----------------+----------------------------+--------------+

      4. Rebalance failed with reason bulk_set_vbucket_state_failed

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. MB-34542 Replica is not resilient to receiving multiple Commits for the same key

          • Test Blocker - QE-related tests can't proceed
          • Closed
          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              ritesh.agarwal Ritesh Agarwal
              ritesh.agarwal Ritesh Agarwal
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty