Details
-
Bug
-
Resolution: Fixed
-
Critical
-
7.2.0
-
7.2.0-5312
-
Untriaged
-
Centos 64-bit
-
-
0
-
Unknown
Description
- Create a 2 KV and 1 index/query node cluster.
- Create a magma bucket(replicas=1) and collections(total collection count including default collections is 51)
- Create 500000000 items sequentially(After creation of few thousands of documents set bucket_history_retention_seconds=259200,bucket_history_retention_bytes=12000000000000)
- Update 500000000 created in above step
- Create 500000000 items sequentially
- Update 500000000 created in above step
- Create five indexes Wait for index building.
- Rebalance in KV with Loading of docs.
- Rebalance Out KV with Loading of docs.(Rebalance completed successfully)
- Rebalance In_Out KV with Loading of docs. (Rebalance completed successfully)
- Swap Rebalance with Loading of docs (Rebalance completed successfully)
- Graceful Failover 1 node and RebalanceOut that node with loading in parallel (Rebalance completed successfully)
- Graceful Failover a node and FullRecovery that node with loading in parallel(Rebalance completed successfully)
- Graeful Failover a node and DeltaRecovery that node with loading in parallel
- Observed Rebalance was hung , After starting rebalance not even a single doc has moved (Wiated for close to one hour before reporting this)
Note:
Cluster is still in same state:
http://172.23.110.64:8091/ui/index.html#/logs?commonBucket=GleamBookUsers0&scenarioZoom=minute&scenario=7udweu5h0
QE-TEST:
guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/ankush_temp_job3.ini -p bucket_storage=magma,bucket_eviction_policy=fullEviction,rerun=False -t aGoodDoctor.Hospital.Murphy.ClusterOpsVolume,nodes_init=2,graceful=True,skip_cleanup=True,num_items=50000000,num_buckets=1,bucket_names=GleamBook,doc_size=1024,bucket_type=membase,eviction_policy=fullEviction,iterations=5,batch_size=1000,sdk_timeout=60,log_level=info,infra_log_level=error,rerun=False,skip_cleanup=True,key_size=18,randomize_doc_size=False,randomize_value=True,assert_crashes_on_load=True,num_collections=50,maxttl=10,num_indexes=5,pc=10,index_nodes=1,cbas_nodes=0,fts_nodes=0,ops_rate=200000,ramQuota=102400,doc_ops=create:update:delete:read,mutation_perc=100,rebl_ops_rate=30000,key_type=RandomKey -m rest'
|
Attachments
For Gerrit Dashboard: MB-56565 | ||||||
---|---|---|---|---|---|---|
# | Subject | Branch | Project | Status | CR | V |
189976,11 | MB-56565: Defer removal of checkpoint until after cursor creation | neo | kv_engine | Status: MERGED | +2 | +1 |
190052,2 | MB-56565, it is approved for 7.2.0. There has not been any 7.2.1 changes in neo for KV yet. Switch kv to neo, so that the changes can be checked in. | master | manifest | Status: MERGED | +2 | +1 |
195643,2 | Merge commit 'b83f4da33' into 'couchbase/master' | master | kv_engine | Status: MERGED | +2 | +1 |