Details
-
Bug
-
Resolution: Fixed
-
Critical
-
Cheshire-Cat
-
7.0.0-4432
-
Untriaged
-
-
1
-
Unknown
Description
Note: This test was intended to run for 5-6 days but failed in 24 hours
Steps:
- Create a 15 node cluster
- Create required buckets and collections.
- Create 100000000 items sequentially
- Rebalance IN with Loading of docs
- Rebalance OUT with Loading of docs
- Rebalance SWAP with Loading of docs
- Rebalance IN/OUT with Loading of docs
- Rebalance OUT/IN with Loading of docs
- Rebalance IN with Loading of docs
- Rebalance OUT with Loading of docs
- Rebalance SWAP with Loading of docs
- Rebalance IN/OUT with Loading of docs
- Rebalance OUT/IN with Loading of docs
- Rebalance IN with Loading of docs
- Rebalance OUT with Loading of docs
- Rebalance SWAP with Loading of docs
- Rebalance IN/OUT with Loading of docs
- Rebalance OUT/IN with Loading of docs
- Rebalance IN with Loading of docs
- Rebalance OUT with Loading of docs
- Rebalance SWAP with Loading of docs
- Rebalance IN/OUT with Loading of docs
- Rebalance OUT/IN with Loading of docs
- Rebalance IN with Loading of docs
Rebalance exited with reason {mover_crashed,
|
{unexpected_exit,
|
{'EXIT',<0.8159.423>,
|
{{{{compromised_reply,{error,timeout}},
|
[{ns_memcached,worker_loop,3,
|
[{file,"src/ns_memcached.erl"},
|
{line,233}]},
|
{proc_lib,init_p_do_apply,3,
|
[{file,"proc_lib.erl"},{line,249}]}]},
|
{gen_server,call,
|
['ns_memcached-GleamBookUsers0',
|
{get_dcp_docs_estimate,353,
|
"replication:ns_1@172.23.121.129->ns_1@172.23.121.116:GleamBookUsers0"},
|
180000]}},
|
{gen_server,call,
|
[{'janitor_agent-GleamBookUsers0',
|
'ns_1@172.23.121.129'},
|
{if_rebalance,<0.20838.393>,
|
{wait_dcp_data_move,
|
['ns_1@172.23.120.170',
|
'ns_1@172.23.121.116'],
|
353}},
|
infinity]}}}}}.
|
Rebalance Operation Id = 4e3ca836c5e2089de5d6ef78519b8dd4
|
Test Details:
guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/test_job_magma.ini -p bucket_storage=couchstore,bucket_eviction_policy=fullEviction,rerun=False -t volumetests.Magma.volume.test_long_rebalance,nodes_init=15,replicas=1,skip_cleanup=True,num_items=500000,num_buckets=1,bucket_names=GleamBook,doc_size=256,bucket_type=membase,compression_mode=off,iterations=10,batch_size=1000,sdk_timeout=60,log_level=debug,infra_log_level=info,rerun=False,skip_cleanup=True,key_size=18,randomize_doc_size=False,randomize_value=True,assert_crashes_on_load=True,maxttl=300,num_buckets=2,num_scopes=10,num_collections=5,doc_ops=expiry,durability=MAJORITY,pc=1,sdk_client_pool=True,get-cbcollect-info=True -m rest'
|