Details
-
Bug
-
Resolution: Duplicate
-
Major
-
Cheshire-Cat
-
7.0.0-2465
-
Untriaged
-
Centos 64-bit
-
-
1
-
Unknown
Description
Steps to repro:
* Created a 3 node cluster(replicas = 1)
|
* Loaded 10 million items of 250 bytes each
|
* Added 3 new nodes and triggerd rebalance
|
* During rebalance new creates and deletions (half of the exisiting elements) was going on
|
* Rebalance exited with mover_Crashed "badmatch"
|
Rebalance error log:
Rebalance exited with reason {mover_crashed,
|
{unexpected_exit,
|
{'EXIT',<0.31095.3>,
|
{{{{badmatch,{error,closed}},
|
[{mc_client_binary,cmd_vocal_recv,5,
|
[{file,"src/mc_client_binary.erl"},
|
{line,155}]},
|
{mc_client_binary,set_cluster_config,4,
|
[{file,"src/mc_client_binary.erl"},
|
{line,743}]},
|
{ns_memcached,do_handle_call,3,
|
[{file,"src/ns_memcached.erl"},
|
{line,559}]},
|
{ns_memcached,worker_loop,3,
|
[{file,"src/ns_memcached.erl"},
|
{line,236}]},
|
{proc_lib,init_p_do_apply,3,
|
[{file,"proc_lib.erl"},{line,247}]}]},
|
{gen_server,call,
|
['ns_memcached-default',
|
{get_dcp_docs_estimate,127,
|
"replication:ns_1@172.23.123.8->ns_1@172.23.123.3:default"},
|
180000]}},
|
{gen_server,call,
|
[{'janitor_agent-default',
|
'ns_1@172.23.123.8'},
|
{if_rebalance,<0.26959.3>,
|
{wait_dcp_data_move,
|
['ns_1@172.23.123.3','ns_1@172.23.123.9'],
|
127}},
|
infinity]}}}}}.
|
Rebalance Operation Id = 3e4ccf2c5cfc66b201a78137a96624b2 hide
|
QE test:
./testrunner -i /tmp/ankush.ini -t rebalance_new.rebalance_in.RebalanceInTests.test_rebalance_in_with_ops,nodes_init=3,nodes_in=3,replicas=1,num_items=10000000,bucket_storage=magma,bucket_eviction_policy=fullEviction,randomize_value=True,vbuckets=128,rerun=false,get-cbcollect-info=True,upgrade_version=7.0.0-2422,skip_cleanup=False
|
Attachments
Issue Links
- duplicates
-
MB-39669 MagmaKVStore::makeCompactionContext: Have not set makeCompactionContextCallback to create a compaction_ctx
- Closed
-
MB-40135 [Magma] MagmaKVStore::readLocalDoc(vbid:vb:38 key:_vbstate not found
- Closed
- relates to
-
MB-41018 Separate out IOError from Rollback not found logging message
- Closed