janitor run during failover might corrupt the configuration

Janitor checks server list for bucket and if server list is [] janitor will consider the bucket to be newly created and will assign all active kv nodes to the server list.

For each bucket, the failover kicks the dead node out of the server list and then runs janitor. The dead node gets deactivated only at the very end of the failover.

So let’s say we have 2 nodes [A, B], B is being failed over, but the bucket resides on [B] only (which might happen due to previously aborted rebalance)

Server list will be changed this way [B] -> [] -> [A, B], and then B is deactivated and we have server list that contains deactivated node which will cause subsequent janitor runs to be aborted with corrupted_server_list error