Details
-
Bug
-
Resolution: Duplicate
-
Major
-
None
-
7.1.0
-
Enterprise Edition 7.1.0 build 1548
-
Untriaged
-
Centos 64-bit
-
1
-
No
Description
Script to Repro
guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/win10-bucket-ops-temp_rebalance_magma.ini rerun=False,disk_optimized_thread_settings=True,get-cbcollect-info=True -t bucket_collections.collections_rebalance.CollectionsRebalance.test_data_load_collections_with_hard_failover_recovery,nodes_init=5,nodes_failover=1,recovery_type=delta,bucket_spec=magma_dgm.1_percent_dgm.5_node_1_replica_magma_512_single_bucket,doc_size=512,randomize_value=True,data_load_stage=during,skip_validations=False'
|
Steps to Repro
1. Create a 5 node cluster.
2021-10-21 06:44:19,450 | test | INFO | pool-3-thread-6 | [table_view:display:72] Rebalance Overview
|
+----------------+----------+-----------------------+----------------+--------------+
|
| Nodes | Services | Version | CPU | Status |
|
+----------------+----------+-----------------------+----------------+--------------+
|
| 172.23.105.164 | kv | 7.1.0-1548-enterprise | 0.739348370927 | Cluster node |
|
| 172.23.105.206 | None | | | <--- IN --- |
|
| 172.23.105.33 | None | | | <--- IN --- |
|
| 172.23.105.36 | None | | | <--- IN --- |
|
| 172.23.106.177 | None | | | <--- IN --- |
|
+----------------+----------+-----------------------+----------------+--------------+
|
2. Create a bucket of 512 MB ram with 1 replica. Create scopes/collections and push the bucket to 1 % DGM
2021-10-21 11:32:26,931 | test | INFO | MainThread | [table_view:display:72] Bucket statistics
|
+---------+-----------+-----------------+----------+------------+-----+-----------+-----------+----------+------------+----------------+
|
| Bucket | Type | Storage Backend | Replicas | Durability | TTL | Items | RAM Quota | RAM Used | Disk Used | ARR |
|
+---------+-----------+-----------------+----------+------------+-----+-----------+-----------+----------+------------+----------------+
|
| default | couchbase | magma | 1 | none | 0 | 244312500 | 2.50 GiB | 1.82 GiB | 200.74 GiB | 0.558682834485 |
|
+---------+-----------+-----------------+----------+------------+-----+-----------+-----------+----------+------------+----------------+
|
3. Hard failover a node
2021-10-21 11:32:30,204 | test | INFO | MainThread | [collections_rebalance:rebalance_operation:721] failing over nodes [ip:172.23.106.177 port:8091 ssh_username:root]
|
4. Do a delta recovery. It hangs.
cbcollect_info attached.
