Details
-
Bug
-
Resolution: Duplicate
-
Critical
-
7.2.6
-
Enterprise Edition 7.2.6 build 8101
-
Untriaged
-
-
0
-
Unknown
Description
rebalance is stuck since 35+ hrs
from
http://172.23.218.153:8091/pools/default/rebalanceProgress
{"status":"running","ns_1@172.23.218.160":{"progress":1},"ns_1@172.23.120.130":{"progress":1},"ns_1@172.23.218.150":{"progress":1},"ns_1@172.23.218.151":{"progress":0},"ns_1@172.23.218.161":{"progress":1.210419999999165e-11},"ns_1@172.23.218.181":{"progress":0},"ns_1@172.23.218.162":{"progress":1.210419999999165e-11},"ns_1@172.23.120.132":{"progress":1},"ns_1@172.23.218.152":{"progress":0},"ns_1@172.23.218.182":{"progress":0},"ns_1@172.23.218.183":{"progress":1},"ns_1@172.23.218.153":{"progress":1},"ns_1@172.23.218.163":{"progress":1},"ns_1@172.23.218.164":{"progress":0},"ns_1@172.23.218.184":{"progress":1},"ns_1@172.23.218.185":{"progress":0},"ns_1@172.23.218.155":{"progress":1},"ns_1@172.23.120.135":{"progress":1},"ns_1@172.23.218.156":{"progress":1},"ns_1@172.23.218.157":{"progress":1},"ns_1@172.23.218.158":{"progress":0},"ns_1@172.23.218.148":{"progress":0},"ns_1@172.23.218.159":{"progress":1},"ns_1@172.23.218.149":{"progress":1}}
|
seems hanged on nodes .161 and .162
does not look like space issue as both these have just around 7% disk used (ssh)
Timestamp around which rebalance started (3 Aug 19:08:51) from .153
[ns_server:info,2024-08-03T19:08:55.729-07:00,ns_1@172.23.218.153:<0.12740.0>:ns_orchestrator:handle_event:497]Skipping janitor in state rebalancing
|
[ns_server:info,2024-08-03T19:08:56.740-07:00,ns_1@172.23.218.153:ns_config_rep<0.10333.0>:ns_config_rep:pull_one_node:421]Pulling config from: 'ns_1@172.23.218.150'
|
[ns_server:info,2024-08-03T19:08:59.704-07:00,ns_1@172.23.218.153:<0.1294.2567>:compaction_daemon:spawn_master_db_compactor:901]Start compaction of master db for bucket ITEM with config:
|
[{parallel_db_and_view_compaction,false},
|
{database_fragmentation_threshold,{30,undefined}},
|
{view_fragmentation_threshold,{30,undefined}}]
|
[ns_server:info,2024-08-03T19:08:59.710-07:00,ns_1@172.23.218.153:<0.27593.2566>:compaction_daemon:spawn_master_db_compactor:901]Start compaction of master db for bucket NEW_ORDER with config:
|
[{parallel_db_and_view_compaction,false},
|
{database_fragmentation_threshold,{30,undefined}},
|
{view_fragmentation_threshold,{30,undefined}}]
|
[ns_server:info,2024-08-03T19:08:59.717-07:00,ns_1@172.23.218.153:<0.6561.2567>:compaction_daemon:spawn_master_db_compactor:901]Start compaction of master db for bucket WAREHOUSE with config:
|
[{parallel_db_and_view_compaction,false},
|
{database_fragmentation_threshold,{30,undefined}},
|
{view_fragmentation_threshold,{30,undefined}}]
|
[ns_server:info,2024-08-03T19:08:59.721-07:00,ns_1@172.23.218.153:<0.6171.2567>:compaction_daemon:spawn_master_db_compactor:901]Start compaction of master db for bucket bucket4 with config:
|
[{parallel_db_and_view_compaction,false},
|
{database_fragmentation_threshold,{30,undefined}},
|
{view_fragmentation_threshold,{30,undefined}}]
|
[ns_server:info,2024-08-03T19:08:59.728-07:00,ns_1@172.23.218.153:<0.26392.2566>:compaction_daemon:spawn_master_db_compactor:901]Start compaction of master db for bucket bucket5 with config:
|
[{parallel_db_and_view_compaction,false},
|
{database_fragmentation_threshold,{30,undefined}},
|
{view_fragmentation_threshold,{30,undefined}}]
|
[ns_server:info,2024-08-03T19:08:59.737-07:00,ns_1@172.23.218.153:<0.994.2567>:compaction_daemon:spawn_master_db_compactor:901]Start compaction of master db for bucket bucket6 with config:
|
[{parallel_db_and_view_compaction,false},
|
{database_fragmentation_threshold,{30,undefined}},
|
{view_fragmentation_threshold,{30,undefined}}]
|
Attachments
Issue Links
- duplicates
-
-
- Closed
-