A race between compaction thread and the memcached thread that changes the vbucket state could fire an assertion crash

If while compaction is running on a particular vbucket, the vbucket state changes from active to replica, and the compaction continues to expire items after the state change, a crash can occur because of an assertion getting fired, when the snapshot marker is received from the new producer based on what has been requested.

The steps would be:

• state of vbucket x changes from active to replica.
• stream request sent from consumer at x to new producer
• Before the first step started, compaction was in progress (The run time of the compaction task would depend on the size of the file)
• If at this point compaction were still running, and after the stream request was sent out - it did manage to expire a good number of items (essentially [expired_item_count + high_seqno] should be greater than the about-to-come-in snapshot end from the producer), then it would very likely fire the assertion while setting the snapshot range that snapshot_end is not greater than or equal to current_high_seqno.