Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-16918

Backup cluster hangs when rebalance is in progress

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Blocker
    • 4.5.0
    • 4.5.0
    • tools
    • Security Level: Public
    • Centos6
      4GB Ram
    • Untriaged
    • No

    Description

      1. Initiate a cluster. Add beer-sample and gamesin-sample to the cluster.
      2. Create a backupset in a machine with CB server (not in the cluster)
      3. Add a node to the cluster and rebalance
      3. While the rebalance is in progress, take a cluster backup
      4. Wait for the backup to be taken. Also wait for the rebalance to happen.

      The backup fails sporadically when there is rollback requested ( Error: A rollback is required. vb 1014 with request (1 164 0 0 164) ). Most of the time, backup recovers from this and moves ahead. But many a time, the backup is struck in this state forever, slowing down the rebalance too in the process.

      To repro the bug, you might need to repeat the steps multiple times. This can be reproduced more consistently while rebalancing a node out.

      To run the test follow these steps:
      1. Clone https://github.com/bharath-gp/testrunner.git
      2. Run tests from this repo.
      4. Have ini file with 4 nodes in them. (Preferably it needs to be VM, like created using vagrants)
      3. Run the following test: ./testrunner -i b/resources/local2.ini -t enterprise_backup_restore.enterprise_backup_restore_test.EnterpriseBackupRestoreTest.test_backup_restore_while_rebalance,items=1000,nodes_init=2,nodes_in=0,nodes_out=1

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              mikew Mike Wiederhold [X] (Inactive)
              bharath.gp Bharath G P
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty