System Test: Reboot destination node, causes "Target database out of sync. Try to increase max_dbs_open at the target's server" 2

Hi Junyi,

On rebooting a node on destination cluster, do we expect that the replication to that node should restart?
XDCR re-replication will depend on whether there are any more open checkpoints to be replicated to. And if the cluster is in steady state .. no incoming mutations/ no incoming/outgoing xdcr traffic , we infer the cluster is in stead state.

Seeing some strange behavior

Source : Sending out some data in bursts
Destination: Doing mainly gets..

This is a unidirectional replication from Source->Destination. The cluster had no incoming load after replicating 33M items. Both source and destination clusters were in steady state.
Rebooted one node on destination.

And the Source replication is failing w/ error "Target database out of sync. Try to increase max_dbs_open at the target's server."

What is the expected behaviour on a "Reboot node" on either source/ destination? Why do we see these bursts of xdcr-traffic on the Source?

Adding screenshots from the source and destination cluster.

LInks :
Source: http://ec2-107-22-40-124.compute-1.amazonaws.com:8091/
Destination: http://ec2-54-235-229-199.compute-1.amazonaws.com:8091/