do not purge deletions that are not yet indexed by views

SUBJ.

"Spec" below:

Compaction/purging

Database compaction is triggered according to compaction settings. Main purpose of compaction is to reclaim disk space used by obsolete mutations.

With 'tombstone purging XXXX' change, database compaction process additionally removes metadata about deletions (or expirations) that occurred long enough time ago. We keep deletions that happened recently (not long enough time ago) so that XDCR can replicate them and so that indexes can remove corresponding views rows. Interval of time between logical deletion of document and physical removal of information about deletion (purging) is called "purge interval".

Purging happens transparently as part of every database compaction.

(Note: currently proposed "what's this" tooltip appears to define quite different thing, you may want to revisit that and work with Karen on correcting it. Particularly it appears to say that purging runs with period of "purge interval" which is clearly incorrect)

If this interval is too short you risk not replicating deletions via XDCR, if this interval is too long you will waste disk space for tombstones (aka document deletions records). Each tombstone occupies about YYYY bytes on disk.

There are 2 exceptions to above rule.

*) shortly after rebalance or failover we will not purge deletions even older deletions. This is in order to account for XDCR inability to maintain it's checkpoints across vbucket moves which happen during rebalance or failover. We give deletions additional "purge interval" after rebalance/failover to give them chance to be XDCR-replicated. Note that xdcr will also lose checkpoints on rebalance/failover of destination cluster, but we do not track that or account for that.

*) indexes keep track information about their most recently indexed mutation. This allows us to avoid purging deletions there are not yet processed by indexes. Our views are normally updated automatically but in case of unusual view update settings or other rare conditions when views are stale, we will prevent database compaction from purging not yet indexed deletions. Development indexes are not automatically updated. And they do not prevent purging. Instead they are automatically reset when view engine detects that they are behind "purge wall".