Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-37175

[backup][2i] cluster fails to restore



    • Untriaged
    • Unknown


      Note: this error is seen as far back as 4471

      Cannot get a local repro, but on jenkins this test fails saying the backup manager failed to restore from the backup (on jenkins the failure is also intermittent, but it happens more often than not)

      ./testrunner -i /tmp/testexec.15656.ini -p dataset=bigdata,GROUP=P2,get-cbcollect-info=True -t 2i.gsi_alter_index_replicas.GSIAlterIndexesTests.test_backup_restore_unhealthy_replica,doc-per-day=1000,nodes_init=4,services_init=kv:n1ql-index-index-index,groups=simple,reset_services=True,GROUP=P2

      This test does the following
      1 kv+n1ql node, 3 index nodes
      it loads 400k docs,
      it creates this index CREATE INDEX idx1 ON default(_id) USING GSI WITH


      now it takes a backup of the cluster as is
      it wipes this cluster and reinitializes a new cluster 1 kv+n1ql node, 3 index nodes
      create default bucket again
      creates the same index again CREATE INDEX idx1 ON default(_id) USING GSI WITH


      failsover a node that contains one of the replicas of the index
      rebalances out this node
      restores from the backup at this point
      restore fails

      We see this error:
      [2019-12-04 15:02:10,361] - [remote_util:2991] INFO - Error restoring cluster: Internal server error while executing "POST" check the server logs for more details

      Looking at the index logs on this timestamp we see this snippet for
      2019-12-04T15:02:10.354-08:00 [Error] requestHandler.makeCreateIndexRequest(): create index request fails. Error=Index default.idx1 already exists

      Will attach logs from all the servers.


        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.



              hemant.rajput Hemant Rajput
              ajay.bhullar Ajay Bhullar
              0 Vote for this issue
              9 Start watching this issue



                Gerrit Reviews

                  There are no open Gerrit changes