Details
-
Bug
-
Resolution: Duplicate
-
Critical
-
7.6.0
-
7.6.0-1854
-
Untriaged
-
Centos 64-bit
-
-
0
-
Unknown
-
KV 2023-4
Description
Step To Recreate:
- Create a 4 node cluster( 172.23.110.64, 172.23.110.65, 172.23.110.67, 172.23.110.68-> all kv nodes)
- Create a bucket with storage backend == couchstore (replicas=1)
- Load 100000000 docs(doc size =1024 bytes)
- Change the storage backend to magma
- While data loading is going on trigger swap rebalance(Out==> 172.23.110.68 and IN==> 172.23.110.128)
- Swap rebalance was successfull newly migrated node had storage back End=magma
- Gracefully failover a node(172.23.110.67) and do a full recovery , after rebalance node has stroagebackEnd= magma
- Hard failover a node (172.23.110.65) and do a full recovery.
- Rebalance exited with reason {mover_crashed, {unexpected_exit, {'EXIT',<0.27602.171>, {{{badmatch, {error, {setup_replications_failed
Looks similar to MB-59826 but since this is a different test, raising a new MB*
Rebalance Failure:
Rebalance exited with reason {mover_crashed,
|
{unexpected_exit,
|
{'EXIT',<0.27602.171>,
|
{{{badmatch,
|
{error,
|
{setup_replications_failed,
|
[{'ns_1@172.23.110.65',
|
{errors,[{134,438}]}}]}}},
|
[{janitor_agent,handle_apply_vbucket_state,
|
2,
|
[{file,"src/janitor_agent.erl"},
|
{line,1068}]},
|
{janitor_agent,
|
apply_vbucket_states_worker_loop,0,
|
[{file,"src/janitor_agent.erl"},
|
{line,1057}]},
|
{proc_lib,init_p,3,
|
[{file,"proc_lib.erl"},{line,225}]}]},
|
{gen_server,call,
|
[{'janitor_agent-GleamBookUsers0',
|
'ns_1@172.23.110.128'},
|
{if_rebalance,<0.6211.161>,
|
{wait_dcp_data_move,
|
['ns_1@172.23.110.65'],
|
437}},
|
infinity]}}}}}.
|
Rebalance Operation Id = 0742e560c5ff746dde2e113d143aeacd
|
QE-TEST:
guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/ankush_temp_job3.ini bucket_storage=couchstore,rerun=False -t aGoodDoctor.Hospital.Murphy.StorageMigrationTestHappyPath,nodes_init=4,graceful=True,skip_cleanup=True,num_items=100000000,num_buckets=1,bucket_names=GleamBook,doc_size=1024,bucket_type=membase,bucket_eviction_policy=valueOnly,iterations=5,batch_size=1000,sdk_timeout=60,log_level=debug,infra_log_level=debug,rerun=False,skip_cleanup=True,key_size=22,assert_crashes_on_load=True,num_collections=50,maxttl=10,num_indexes=0,pc=10,indexer_mem_quota=0,index_nodes=0,cbas_nodes=0,fts_nodes=0,ops_rate=200000,ramQuota=10240,doc_ops=create:update:delete:read,mutation_perc=100,rebl_ops_rate=50000,key_type=RandomKey,revert_migration=True'
|
Attachments
Issue Links
- duplicates
-
-
- Closed
-