Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-49410

[Magma , 4TB Volume Test] Rebalance In failed with error mover_crashed,child_interrupted, dcp_replicatorspawn_and_wait 1

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Critical
    • None
    • 7.1.0
    • couchbase-bucket
    • 7.1.0-1605

    Description

      Steps:
      1. Create a 4 node cluster
      2. Create buckets and 50 collections.
      3. Create 20000000 items in each collection:

      Read Start: 0
      Read End: 0
      Update Start: 0
      Update End: 0
      Expiry Start: 0
      Expiry End: 0
      Delete Start: 0
      Delete End: 0
      Create Start: 0
      Create End: 20000000
      Final Start: 0
      Final End: 20000000
      

      4. Update 20000000 keys to create 50 percent fragmentation:

      Read Start: 0
      Read End: 0
      Update Start: 0
      Update End: 20000000
      Expiry Start: 0
      Expiry End: 0
      Delete Start: 0
      Delete End: 0
      Create Start: 0
      Create End: 0
      Final Start: 0
      Final End: 20000000
      

      5. Create another 20000000 items:

      Read Start: 0
      Read End: 0
      Update Start: 0
      Update End: 0
      Expiry Start: 0
      Expiry End: 0
      Delete Start: 0
      Delete End: 0
      Create Start: 20000000
      Create End: 40000000
      Final Start: 20000000
      Final End: 40000000
      

      6. Update 20000000 keys (created in step 5) to maintain 50 percent fragmentation

      Read Start: 0
      Read End: 0
      Update Start: 20000000
      Update End: 40000000
      Expiry Start: 0
      Expiry End: 0
      Delete Start: 0
      Delete End: 0
      Create Start: 0
      Create End: 0
      Final Start: 20000000
      Final End: 40000000
      

      7. Start ASYNC load:

      Read Start: 0
      Read End: 20000000
      Update Start: 0
      Update End: 20000000
      Expiry Start: 0
      Expiry End: 0
      Delete Start: 20000000
      Delete End: 40000000
      Create Start: 40000000
      Create End: 60000000
      Final Start: 40000000
      Final End: 60000000
      

      8. Rebalance IN with Loading of docs in step 7
      Rebalance failed with error mover_crashed,child_interrupted, dcp_replicatorspawn_and_wait 1

      Rebalance exited with reason {mover_crashed,
      {unexpected_exit,
      {'EXIT',<0.9052.162>,
      {{{child_interrupted,
      {'EXIT',<33584.10425.90>,socket_closed}},
      [{dcp_replicator,spawn_and_wait,1,
      [{file,"src/dcp_replicator.erl"},
      {line,308}]},
      {dcp_replicator,handle_call,3,
      [{file,"src/dcp_replicator.erl"},
      {line,129}]},
      {gen_server,try_handle_call,4,
      [{file,"gen_server.erl"},{line,721}]},
      {gen_server,handle_msg,6,
      [{file,"gen_server.erl"},{line,750}]},
      {proc_lib,init_p_do_apply,3,
      [{file,"proc_lib.erl"},{line,226}]}]},
      {gen_server,call,
      [{'janitor_agent-GleamBookUsers0',
      'ns_1@172.23.107.217'},
      {if_rebalance,<0.4145.144>,
      {dcp_takeover,'ns_1@172.23.107.223',426}},
      infinity]}}}}}.
      Rebalance Operation Id = cfe55ab4e66999e8863c486120f37d92
      

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            ankush.sharma Ankush Sharma
            ankush.sharma Ankush Sharma
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty