Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-40129

[Magma]MagmaKVStore::makeCompactionContext vb:76 failed to read vbstate from disk

    XMLWordPrintable

Details

    Description

      Steps to repro:

      * Created a 3 node cluster(replicas = 1)
      * Loaded 10 million items of 250 bytes each
      * Added 3 new nodes and triggerd rebalance
      * During rebalance new creates and deletions (half of the exisiting elements) was going on
      * Rebalance failed with "wait_seqno_persisted-failed"
      

      Rebalance error log:

      Rebalance exited with reason {mover_crashed,
      {unexpected_exit,
      {'EXIT',<0.11003.1>,
      {{wait_seqno_persisted_failed,"default",113,
      103335,
      [{'ns_1@172.23.123.4',
      {'EXIT',
      {socket_closed,
      {gen_server,call,
      [{'janitor_agent-default',
      'ns_1@172.23.123.4'},
      {if_rebalance,<0.30675.0>,
      {wait_seqno_persisted,113,103335}},
      infinity]}}}}]},
      [{ns_single_vbucket_mover,
      '-wait_seqno_persisted_many/5-fun-2-',5,
      [{file,"src/ns_single_vbucket_mover.erl"},
      {line,488}]},
      {proc_lib,init_p,3,
      [{file,"proc_lib.erl"},{line,232}]}]}}}}.
      Rebalance Operation Id = 4505ce790274d9ac95bf8ec4df85672e
      

      QE test:

      ./testrunner -i /tmp/ankush.ini  -t rebalance_new.rebalance_in.RebalanceInTests.test_rebalance_in_with_ops,nodes_init=3,nodes_in=3,replicas=1,num_items=10000000,bucket_storage=magma,bucket_eviction_policy=fullEviction,randomize_value=True,vbuckets=128,rerun=false,get-cbcollect-info=True,upgrade_version=7.0.0-2422,skip_cleanup=False
      

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              ankush.sharma Ankush Sharma
              ankush.sharma Ankush Sharma
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty