[System test]Indexer Rebalance stuck for 11+hrs in system test

In system test running on 7.2.3-6710 it was observed that indexer rebalance is stuck for 11+ hrs.
Jenkins job - http://qa.sc.couchbase.com/job/component_systest_launcher_2/802/console 

Cluster config - kv-4 , index-5 and query-2

Logs 

iteration 20

url : https://cb-engineering.s3.amazonaws.com/yash-test/collectinfo-2024-02-15T073842-ns_1%40172.23.106.180.zip
url : https://cb-engineering.s3.amazonaws.com/yash-test/collectinfo-2024-02-15T073842-ns_1%40172.23.106.183.zip
url : https://cb-engineering.s3.amazonaws.com/yash-test/collectinfo-2024-02-15T073842-ns_1%40172.23.106.189.zip
url : https://cb-engineering.s3.amazonaws.com/yash-test/collectinfo-2024-02-15T073842-ns_1%40172.23.106.246.zip
url : https://cb-engineering.s3.amazonaws.com/yash-test/collectinfo-2024-02-15T073842-ns_1%40172.23.107.114.zip
url : https://cb-engineering.s3.amazonaws.com/yash-test/collectinfo-2024-02-15T073842-ns_1%40172.23.107.20.zip
url : https://cb-engineering.s3.amazonaws.com/yash-test/collectinfo-2024-02-15T073842-ns_1%40172.23.107.23.zip
url : https://cb-engineering.s3.amazonaws.com/yash-test/collectinfo-2024-02-15T073842-ns_1%40172.23.107.82.zip
url : https://cb-engineering.s3.amazonaws.com/yash-test/collectinfo-2024-02-15T073842-ns_1%40172.23.107.98.zip
url : https://cb-engineering.s3.amazonaws.com/yash-test/collectinfo-2024-02-15T073842-ns_1%40172.23.97.124.zip
url : https://cb-engineering.s3.amazonaws.com/yash-test/collectinfo-2024-02-15T073842-ns_1%40172.23.97.139.zip

iteration 18

url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707974144/collectinfo-2024-02-15T052640-ns_1%40172.23.106.180.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707974144/collectinfo-2024-02-15T052640-ns_1%40172.23.106.183.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707974144/collectinfo-2024-02-15T052640-ns_1%40172.23.106.189.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707974144/collectinfo-2024-02-15T052640-ns_1%40172.23.106.246.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707974144/collectinfo-2024-02-15T052640-ns_1%40172.23.107.114.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707974144/collectinfo-2024-02-15T052640-ns_1%40172.23.107.20.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707974144/collectinfo-2024-02-15T052640-ns_1%40172.23.107.23.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707974144/collectinfo-2024-02-15T052640-ns_1%40172.23.107.82.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707974144/collectinfo-2024-02-15T052640-ns_1%40172.23.107.98.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707974144/collectinfo-2024-02-15T052640-ns_1%40172.23.97.124.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707974144/collectinfo-2024-02-15T052640-ns_1%40172.23.97.139.zip

iteration 15
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707961159/collectinfo-2024-02-15T015104-ns_1%40172.23.106.180.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707961159/collectinfo-2024-02-15T015104-ns_1%40172.23.106.183.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707961159/collectinfo-2024-02-15T015104-ns_1%40172.23.106.189.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707961159/collectinfo-2024-02-15T015104-ns_1%40172.23.106.246.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707961159/collectinfo-2024-02-15T015104-ns_1%40172.23.107.114.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707961159/collectinfo-2024-02-15T015104-ns_1%40172.23.107.20.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707961159/collectinfo-2024-02-15T015104-ns_1%40172.23.107.23.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707961159/collectinfo-2024-02-15T015104-ns_1%40172.23.107.82.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707961159/collectinfo-2024-02-15T015104-ns_1%40172.23.107.98.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707961159/collectinfo-2024-02-15T015104-ns_1%40172.23.97.124.zip
url : https://cb-jira.s3.us-east-2.amazonaws.com/logs/systestmon-1707961159/collectinfo-2024-02-15T015104-ns_1%40172.23.97.139.zip