This is an extension of
MB-30019. Checkpoint expelling reduced the impact of replica checkpoint memory after data ingestion by removing no longer necessary items from the checkpoints. Expelling does not touch the keyIndexes in Checkpoints though which is the majority of the "overhead" memory. The issue still exists and could causes clusters to livelock with mem used between the new mutation threshold and replication threshold as the majority of memory belongs to replica checkpoint overheads. Attempts were made under MB-30019 to close replica checkpoints at snapshot end which should be fine for the sake of consistency but does make replica checkpoint cutting diverge from the active (something that we have historically avoided). These attempts were abandoned due to performance reasons and could be revisited.