[CDC] Disk Backfill triggered removal of the next required checkpoint - was: Rebalance stuck/hung for more than 5 hours

Steps:

1. Create a 2 KV and 1 index/query node cluster.
2. Create a magma bucket(replicas=1) and collections(total collection count including default collections is 51)
3. Create 500000000 items sequentially(After creation of few thousands of documents set bucket_history_retention_seconds=259200,bucket_history_retention_bytes=12000000000000)
4. Update 500000000 created in above step
5. Create 500000000 items sequentially
6. Update 500000000 created in above step
7. Create five indexes Wait for index building.
8. Rebalance in KV with Loading of docs.
9. Rebalance was stuck for more than five hours , not even a single document moved (Attached are screen shots for same, though I took screen shots bit late, but it is evident from screen shots as well no document got moved for approx 2 hours.

Note:
It has been observed earlier on 7.2.0-5298 as well, Rebalance was stuck for 8 hours last time and after stopping data loading eventually it got succeed, But we lost the logs last time(because of jenkins failure)

Cluster is still in same state:
http://172.23.110.64:8091/ui/index.html#/overview/stats?commonBucket=GleamBookUsers0&scenarioZoom=minute&scenario=enhoj63l3&statsHostname=all

QE-TEST:

guides/gradlew --refresh-dependencies testrunner -P jython=/opt/jython/bin/jython -P 'args=-i /tmp/ankush_temp_job3.ini -p bucket_storage=magma,bucket_eviction_policy=fullEviction,rerun=False -t aGoodDoctor.Hospital.Murphy.ClusterOpsVolume,nodes_init=2,graceful=True,skip_cleanup=True,num_items=50000000,num_buckets=1,bucket_names=GleamBook,doc_size=1024,bucket_type=membase,eviction_policy=fullEviction,iterations=5,batch_size=1000,sdk_timeout=60,log_level=info,infra_log_level=error,rerun=False,skip_cleanup=True,key_size=18,randomize_doc_size=False,randomize_value=True,assert_crashes_on_load=True,num_collections=50,maxttl=10,num_indexes=5,pc=10,index_nodes=1,cbas_nodes=0,fts_nodes=0,ops_rate=200000,ramQuota=102400,doc_ops=create:update:delete:read,mutation_perc=100,rebl_ops_rate=30000,key_type=RandomKey -m rest'