Replication is stuck post rebalance of orchestrator (master): Active master and replica's are not in sync, resulting in data loss (0.02%)

The following test seems to be failing, becoz replication does not finish.
./testrunner my_vm.ini -t swaprebalance.SwapRebalanceBasicTests.do_test,replica=2,num-buckets=4,num-swap=1,swap-orchestrator=True

This tests swap rebalances the orchestrator node out of the cluster with a brand new node.

The rebalance completed from the user point of view.

2012-04-16 17:46:33,088 - root - INFO - delta : 163 missing_percentage : 0.066867222754 replica_factor : 2
2012-04-16 17:46:35,342 - root - INFO - curr_items from 10.1.3.115:8091 : 2422
2012-04-16 17:46:35,342 - root - INFO - vb_pending_num from 10.1.3.115:8091 : 0
2012-04-16 17:46:35,342 - root - INFO - vb_active_num from 10.1.3.115:8091 : 21
2012-04-16 17:46:35,342 - root - INFO - vb_replica_num from 10.1.3.115:8091 : 43
2012-04-16 17:46:35,342 - root - INFO - curr_items from 10.1.3.116:8091 : 2329
2012-04-16 17:46:35,342 - root - INFO - vb_pending_num from 10.1.3.116:8091 : 0
2012-04-16 17:46:35,343 - root - INFO - vb_active_num from 10.1.3.116:8091 : 21
2012-04-16 17:46:35,343 - root - INFO - vb_replica_num from 10.1.3.116:8091 : 43
2012-04-16 17:46:35,343 - root - INFO - curr_items from 10.1.3.117:8091 : 2562
2012-04-16 17:46:35,343 - root - INFO - vb_pending_num from 10.1.3.117:8091 : 0
2012-04-16 17:46:35,343 - root - INFO - vb_active_num from 10.1.3.117:8091 : 22
2012-04-16 17:46:35,343 - root - INFO - vb_replica_num from 10.1.3.117:8091 : 42
2012-04-16 17:46:35,343 - root - INFO - summation of vb_active_num : 64 vb_pending_num : 0 vb_replica_num : 128
2012-04-16 17:46:35,344 - root - INFO - sum : 7313 and sum * replica_factor (2) : 21939
2012-04-16 17:46:35,412 - root - INFO - master_stats : 21776
2012-04-16 17:46:35,412 - root - INFO - delta : 163 missing_percentage : 0.066867222754 replica_factor : 2
2012-04-16 17:46:37,668 - root - INFO - curr_items from 10.1.3.115:8091 : 2422
2012-04-16 17:46:37,668 - root - INFO - vb_pending_num from 10.1.3.115:8091 : 0
2012-04-16 17:46:37,668 - root - INFO - vb_active_num from 10.1.3.115:8091 : 21
2012-04-16 17:46:37,668 - root - INFO - vb_replica_num from 10.1.3.115:8091 : 43
2012-04-16 17:46:37,669 - root - INFO - curr_items from 10.1.3.116:8091 : 2329
2012-04-16 17:46:37,669 - root - INFO - vb_pending_num from 10.1.3.116:8091 : 0
2012-04-16 17:46:37,669 - root - INFO - vb_active_num from 10.1.3.116:8091 : 21
2012-04-16 17:46:37,669 - root - INFO - vb_replica_num from 10.1.3.116:8091 : 43
2012-04-16 17:46:37,669 - root - INFO - curr_items from 10.1.3.117:8091 : 2562
2012-04-16 17:46:37,669 - root - INFO - vb_pending_num from 10.1.3.117:8091 : 0
2012-04-16 17:46:37,669 - root - INFO - vb_active_num from 10.1.3.117:8091 : 22
2012-04-16 17:46:37,669 - root - INFO - vb_replica_num from 10.1.3.117:8091 : 42
2012-04-16 17:46:37,669 - root - INFO - summation of vb_active_num : 64 vb_pending_num : 0 vb_replica_num : 128
2012-04-16 17:46:37,669 - root - INFO - sum : 7313 and sum * replica_factor (2) : 21939
2012-04-16 17:46:37,742 - root - INFO - master_stats : 21776
2012-04-16 17:46:37,742 - root - INFO - delta : 163 missing_percentage : 0.066867222754 replica_factor : 2
2012-04-16 17:46:39,994 - root - INFO - curr_items from 10.1.3.115:8091 : 2422
2012-04-16 17:46:39,994 - root - INFO - vb_pending_num from 10.1.3.115:8091 : 0
2012-04-16 17:46:39,994 - root - INFO - vb_active_num from 10.1.3.115:8091 : 21
2012-04-16 17:46:39,995 - root - INFO - vb_replica_num from 10.1.3.115:8091 : 43
2012-04-16 17:46:39,995 - root - INFO - curr_items from 10.1.3.116:8091 : 2329
2012-04-16 17:46:39,995 - root - INFO - vb_pending_num from 10.1.3.116:8091 : 0
2012-04-16 17:46:39,995 - root - INFO - vb_active_num from 10.1.3.116:8091 : 21
2012-04-16 17:46:39,995 - root - INFO - vb_replica_num from 10.1.3.116:8091 : 43
2012-04-16 17:46:39,995 - root - INFO - curr_items from 10.1.3.117:8091 : 2562
2012-04-16 17:46:39,995 - root - INFO - vb_pending_num from 10.1.3.117:8091 : 0
2012-04-16 17:46:39,995 - root - INFO - vb_active_num from 10.1.3.117:8091 : 22
2012-04-16 17:46:39,995 - root - INFO - vb_replica_num from 10.1.3.117:8091 : 42
2012-04-16 17:46:39,995 - root - INFO - summation of vb_active_num : 64 vb_pending_num : 0 vb_replica_num : 128
2012-04-16 17:46:39,995 - root - INFO - sum : 7313 and sum * replica_factor (2) : 21939
2012-04-16 17:46:40,062 - root - INFO - master_stats : 21776
2012-04-16 17:46:40,062 - root - INFO - delta : 163 missing_percentage : 0.066867222754 replica_factor : 2

115
ep_tap_ack_grace_period: 300
ep_tap_ack_interval: 1000
ep_tap_ack_window_size: 10
ep_tap_backoff_period: 5
ep_tap_bg_fetch_requeued: 0
ep_tap_bg_fetched: 0
ep_tap_bg_max_pending: 500
ep_tap_count: 4
ep_tap_deletes: 5710
ep_tap_fg_fetched: 75181
ep_tap_keepalive: 300
ep_tap_noop_interval: 20
ep_tap_queue_backfillremaining: 0
ep_tap_queue_backoff: 0
ep_tap_queue_drain: 45074
ep_tap_queue_fill: 0
ep_tap_queue_itemondisk: 0
ep_tap_throttle_queue_cap: 1000000
ep_tap_throttle_threshold: 90
ep_tap_throttled: 0
ep_tap_total_backlog_size: 0
ep_tap_total_fetched: 82482
ep_tap_total_queue: 0
eq_tapq:anon_696:connected: true
eq_tapq:anon_696:created: 121
eq_tapq:anon_696:num_checkpoint_end: 21
eq_tapq:anon_696:num_checkpoint_end_failed: 0
eq_tapq:anon_696:num_checkpoint_start: 22
eq_tapq:anon_696:num_checkpoint_start_failed: 0
eq_tapq:anon_696:num_delete: 127
eq_tapq:anon_696:num_delete_failed: 0
eq_tapq:anon_696:num_flush: 0
eq_tapq:anon_696:num_flush_failed: 0
eq_tapq:anon_696:num_mutation: 1277
eq_tapq:anon_696:num_mutation_failed: 0
eq_tapq:anon_696:num_opaque: 4
eq_tapq:anon_696:num_opaque_failed: 0
eq_tapq:anon_696:num_unknown: 0
eq_tapq:anon_696:num_vbucket_set: 0
eq_tapq:anon_696:num_vbucket_set_failed: 0
eq_tapq:anon_696:pending_disconnect: false
eq_tapq:anon_696:reserved: 0
eq_tapq:anon_696:supports_ack: true
eq_tapq:anon_696:type: consumer
eq_tapq:anon_705:connected: true
eq_tapq:anon_705:created: 122
eq_tapq:anon_705:num_checkpoint_end: 22
eq_tapq:anon_705:num_checkpoint_end_failed: 0
eq_tapq:anon_705:num_checkpoint_start: 23
eq_tapq:anon_705:num_checkpoint_start_failed: 0
eq_tapq:anon_705:num_delete: 98
eq_tapq:anon_705:num_delete_failed: 0
eq_tapq:anon_705:num_flush: 0
eq_tapq:anon_705:num_flush_failed: 0
eq_tapq:anon_705:num_mutation: 1086
eq_tapq:anon_705:num_mutation_failed: 0
eq_tapq:anon_705:num_opaque: 4
eq_tapq:anon_705:num_opaque_failed: 0
eq_tapq:anon_705:num_unknown: 0
eq_tapq:anon_705:num_vbucket_set: 0
eq_tapq:anon_705:num_vbucket_set_failed: 0
eq_tapq:anon_705:pending_disconnect: false
eq_tapq:anon_705:reserved: 0
eq_tapq:anon_705:supports_ack: true
eq_tapq:anon_705:type: consumer
eq_tapq:replication_ns_1@10.1.3.115:ack_log_size: 0
eq_tapq:replication_ns_1@10.1.3.115:ack_playback_size: 0
eq_tapq:replication_ns_1@10.1.3.115:ack_seqno: 35928
eq_tapq:replication_ns_1@10.1.3.115:ack_window_full: false
eq_tapq:replication_ns_1@10.1.3.115:backfill_completed: true
eq_tapq:replication_ns_1@10.1.3.115:bg_backlog_size: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_jobs_completed: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_jobs_issued: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_queued: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_result_size: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_results: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_wait_for_results: false
eq_tapq:replication_ns_1@10.1.3.115:complete: false
eq_tapq:replication_ns_1@10.1.3.115:connected: true
eq_tapq:replication_ns_1@10.1.3.115:created: 62
eq_tapq:replication_ns_1@10.1.3.115:disconnects: 42
eq_tapq:replication_ns_1@10.1.3.115:empty: true
eq_tapq:replication_ns_1@10.1.3.115:flags: 85 (ack,backfill,vblist,checkpoints)
eq_tapq:replication_ns_1@10.1.3.115:has_item: false
eq_tapq:replication_ns_1@10.1.3.115:has_queued_item: false
eq_tapq:replication_ns_1@10.1.3.115:idle: true
eq_tapq:replication_ns_1@10.1.3.115:num_tap_nack: 0
eq_tapq:replication_ns_1@10.1.3.115:num_tap_tmpfail_survivors: 0
eq_tapq:replication_ns_1@10.1.3.115:paused: 1
eq_tapq:replication_ns_1@10.1.3.115:pending_backfill: false
eq_tapq:replication_ns_1@10.1.3.115:pending_disconnect: false
eq_tapq:replication_ns_1@10.1.3.115:pending_disk_backfill: false
eq_tapq:replication_ns_1@10.1.3.115:qlen: 0
eq_tapq:replication_ns_1@10.1.3.115:qlen_high_pri: 0
eq_tapq:replication_ns_1@10.1.3.115:qlen_low_pri: 0
eq_tapq:replication_ns_1@10.1.3.115:queue_backfillremaining: 0
eq_tapq:replication_ns_1@10.1.3.115:queue_backoff: 0
eq_tapq:replication_ns_1@10.1.3.115:queue_drain: 35235
eq_tapq:replication_ns_1@10.1.3.115:queue_fill: 0
eq_tapq:replication_ns_1@10.1.3.115:queue_itemondisk: 0
eq_tapq:replication_ns_1@10.1.3.115:queue_memory: 0
eq_tapq:replication_ns_1@10.1.3.115:rec_fetched: 35740
eq_tapq:replication_ns_1@10.1.3.115:reconnects: 42
eq_tapq:replication_ns_1@10.1.3.115:recv_ack_seqno: 35927
eq_tapq:replication_ns_1@10.1.3.115:reserved: 1
eq_tapq:replication_ns_1@10.1.3.115:seqno_ack_requested: 35927
eq_tapq:replication_ns_1@10.1.3.115:supports_ack: true
eq_tapq:replication_ns_1@10.1.3.115:suspended: false
eq_tapq:replication_ns_1@10.1.3.115:total_backlog_size: 0
eq_tapq:replication_ns_1@10.1.3.115:total_noops: 54
eq_tapq:replication_ns_1@10.1.3.115:type: producer
eq_tapq:replication_ns_1@10.1.3.115:vb_filter:

{ [11,21], [54,63] }
eq_tapq:replication_ns_1@10.1.3.115:vb_filters: 21
eq_tapq:replication_ns_1@10.1.3.117:ack_log_size: 0
eq_tapq:replication_ns_1@10.1.3.117:ack_playback_size: 0
eq_tapq:replication_ns_1@10.1.3.117:ack_seqno: 10211
eq_tapq:replication_ns_1@10.1.3.117:ack_window_full: false
eq_tapq:replication_ns_1@10.1.3.117:backfill_completed: true
eq_tapq:replication_ns_1@10.1.3.117:bg_backlog_size: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_jobs_completed: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_jobs_issued: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_queued: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_result_size: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_results: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_wait_for_results: false
eq_tapq:replication_ns_1@10.1.3.117:complete: false
eq_tapq:replication_ns_1@10.1.3.117:connected: true
eq_tapq:replication_ns_1@10.1.3.117:created: 111
eq_tapq:replication_ns_1@10.1.3.117:disconnects: 20
eq_tapq:replication_ns_1@10.1.3.117:empty: true
eq_tapq:replication_ns_1@10.1.3.117:flags: 85 (ack,backfill,vblist,checkpoints)
eq_tapq:replication_ns_1@10.1.3.117:has_item: false
eq_tapq:replication_ns_1@10.1.3.117:has_queued_item: false
eq_tapq:replication_ns_1@10.1.3.117:idle: true
eq_tapq:replication_ns_1@10.1.3.117:num_tap_nack: 0
eq_tapq:replication_ns_1@10.1.3.117:num_tap_tmpfail_survivors: 0
eq_tapq:replication_ns_1@10.1.3.117:paused: 1
eq_tapq:replication_ns_1@10.1.3.117:pending_backfill: false
eq_tapq:replication_ns_1@10.1.3.117:pending_disconnect: false
eq_tapq:replication_ns_1@10.1.3.117:pending_disk_backfill: false
eq_tapq:replication_ns_1@10.1.3.117:qlen: 0
eq_tapq:replication_ns_1@10.1.3.117:qlen_high_pri: 0
eq_tapq:replication_ns_1@10.1.3.117:qlen_low_pri: 0
eq_tapq:replication_ns_1@10.1.3.117:queue_backfillremaining: 0
eq_tapq:replication_ns_1@10.1.3.117:queue_backoff: 0
eq_tapq:replication_ns_1@10.1.3.117:queue_drain: 9839
eq_tapq:replication_ns_1@10.1.3.117:queue_fill: 0
eq_tapq:replication_ns_1@10.1.3.117:queue_itemondisk: 0
eq_tapq:replication_ns_1@10.1.3.117:queue_memory: 0
eq_tapq:replication_ns_1@10.1.3.117:rec_fetched: 10110
eq_tapq:replication_ns_1@10.1.3.117:reconnects: 20
eq_tapq:replication_ns_1@10.1.3.117:recv_ack_seqno: 10210
eq_tapq:replication_ns_1@10.1.3.117:reserved: 1
eq_tapq:replication_ns_1@10.1.3.117:seqno_ack_requested: 10210
eq_tapq:replication_ns_1@10.1.3.117:supports_ack: true
eq_tapq:replication_ns_1@10.1.3.117:suspended: false
eq_tapq:replication_ns_1@10.1.3.117:total_backlog_size: 0
eq_tapq:replication_ns_1@10.1.3.117:total_noops: 52
eq_tapq:replication_ns_1@10.1.3.117:type: producer
eq_tapq:replication_ns_1@10.1.3.117:vb_filter: { [33,53] }
eq_tapq:replication_ns_1@10.1.3.117:vb_filters: 21
116
ep_tap_ack_grace_period: 300
ep_tap_ack_interval: 1000
ep_tap_ack_window_size: 10
ep_tap_backoff_period: 5
ep_tap_bg_fetch_requeued: 0
ep_tap_bg_fetched: 0
ep_tap_bg_max_pending: 500
ep_tap_count: 4
ep_tap_deletes: 5710
ep_tap_fg_fetched: 75181
ep_tap_keepalive: 300
ep_tap_noop_interval: 20
ep_tap_queue_backfillremaining: 0
ep_tap_queue_backoff: 0
ep_tap_queue_drain: 45074
ep_tap_queue_fill: 0
ep_tap_queue_itemondisk: 0
ep_tap_throttle_queue_cap: 1000000
ep_tap_throttle_threshold: 90
ep_tap_throttled: 0
ep_tap_total_backlog_size: 0
ep_tap_total_fetched: 82482
ep_tap_total_queue: 0
eq_tapq:anon_696:connected: true
eq_tapq:anon_696:created: 121
eq_tapq:anon_696:num_checkpoint_end: 21
eq_tapq:anon_696:num_checkpoint_end_failed: 0
eq_tapq:anon_696:num_checkpoint_start: 22
eq_tapq:anon_696:num_checkpoint_start_failed: 0
eq_tapq:anon_696:num_delete: 127
eq_tapq:anon_696:num_delete_failed: 0
eq_tapq:anon_696:num_flush: 0
eq_tapq:anon_696:num_flush_failed: 0
eq_tapq:anon_696:num_mutation: 1277
eq_tapq:anon_696:num_mutation_failed: 0
eq_tapq:anon_696:num_opaque: 4
eq_tapq:anon_696:num_opaque_failed: 0
eq_tapq:anon_696:num_unknown: 0
eq_tapq:anon_696:num_vbucket_set: 0
eq_tapq:anon_696:num_vbucket_set_failed: 0
eq_tapq:anon_696:pending_disconnect: false
eq_tapq:anon_696:reserved: 0
eq_tapq:anon_696:supports_ack: true
eq_tapq:anon_696:type: consumer
eq_tapq:anon_705:connected: true
eq_tapq:anon_705:created: 122
eq_tapq:anon_705:num_checkpoint_end: 22
eq_tapq:anon_705:num_checkpoint_end_failed: 0
eq_tapq:anon_705:num_checkpoint_start: 23
eq_tapq:anon_705:num_checkpoint_start_failed: 0
eq_tapq:anon_705:num_delete: 98
eq_tapq:anon_705:num_delete_failed: 0
eq_tapq:anon_705:num_flush: 0
eq_tapq:anon_705:num_flush_failed: 0
eq_tapq:anon_705:num_mutation: 1086
eq_tapq:anon_705:num_mutation_failed: 0
eq_tapq:anon_705:num_opaque: 4
eq_tapq:anon_705:num_opaque_failed: 0
eq_tapq:anon_705:num_unknown: 0
eq_tapq:anon_705:num_vbucket_set: 0
eq_tapq:anon_705:num_vbucket_set_failed: 0
eq_tapq:anon_705:pending_disconnect: false
eq_tapq:anon_705:reserved: 0
eq_tapq:anon_705:supports_ack: true
eq_tapq:anon_705:type: consumer
eq_tapq:replication_ns_1@10.1.3.115:ack_log_size: 0
eq_tapq:replication_ns_1@10.1.3.115:ack_playback_size: 0
eq_tapq:replication_ns_1@10.1.3.115:ack_seqno: 35928
eq_tapq:replication_ns_1@10.1.3.115:ack_window_full: false
eq_tapq:replication_ns_1@10.1.3.115:backfill_completed: true
eq_tapq:replication_ns_1@10.1.3.115:bg_backlog_size: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_jobs_completed: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_jobs_issued: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_queued: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_result_size: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_results: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_wait_for_results: false
eq_tapq:replication_ns_1@10.1.3.115:complete: false
eq_tapq:replication_ns_1@10.1.3.115:connected: true
eq_tapq:replication_ns_1@10.1.3.115:created: 62
eq_tapq:replication_ns_1@10.1.3.115:disconnects: 42
eq_tapq:replication_ns_1@10.1.3.115:empty: true
eq_tapq:replication_ns_1@10.1.3.115:flags: 85 (ack,backfill,vblist,checkpoints)
eq_tapq:replication_ns_1@10.1.3.115:has_item: false
eq_tapq:replication_ns_1@10.1.3.115:has_queued_item: false
eq_tapq:replication_ns_1@10.1.3.115:idle: true
eq_tapq:replication_ns_1@10.1.3.115:num_tap_nack: 0
eq_tapq:replication_ns_1@10.1.3.115:num_tap_tmpfail_survivors: 0
eq_tapq:replication_ns_1@10.1.3.115:paused: 1
eq_tapq:replication_ns_1@10.1.3.115:pending_backfill: false
eq_tapq:replication_ns_1@10.1.3.115:pending_disconnect: false
eq_tapq:replication_ns_1@10.1.3.115:pending_disk_backfill: false
eq_tapq:replication_ns_1@10.1.3.115:qlen: 0
eq_tapq:replication_ns_1@10.1.3.115:qlen_high_pri: 0
eq_tapq:replication_ns_1@10.1.3.115:qlen_low_pri: 0
eq_tapq:replication_ns_1@10.1.3.115:queue_backfillremaining: 0
eq_tapq:replication_ns_1@10.1.3.115:queue_backoff: 0
eq_tapq:replication_ns_1@10.1.3.115:queue_drain: 35235
eq_tapq:replication_ns_1@10.1.3.115:queue_fill: 0
eq_tapq:replication_ns_1@10.1.3.115:queue_itemondisk: 0
eq_tapq:replication_ns_1@10.1.3.115:queue_memory: 0
eq_tapq:replication_ns_1@10.1.3.115:rec_fetched: 35740
eq_tapq:replication_ns_1@10.1.3.115:reconnects: 42
eq_tapq:replication_ns_1@10.1.3.115:recv_ack_seqno: 35927
eq_tapq:replication_ns_1@10.1.3.115:reserved: 1
eq_tapq:replication_ns_1@10.1.3.115:seqno_ack_requested: 35927
eq_tapq:replication_ns_1@10.1.3.115:supports_ack: true
eq_tapq:replication_ns_1@10.1.3.115:suspended: false
eq_tapq:replication_ns_1@10.1.3.115:total_backlog_size: 0
eq_tapq:replication_ns_1@10.1.3.115:total_noops: 54
eq_tapq:replication_ns_1@10.1.3.115:type: producer
eq_tapq:replication_ns_1@10.1.3.115:vb_filter: { [11,21], [54,63] }

eq_tapq:replication_ns_1@10.1.3.115:vb_filters: 21
eq_tapq:replication_ns_1@10.1.3.117:ack_log_size: 0
eq_tapq:replication_ns_1@10.1.3.117:ack_playback_size: 0
eq_tapq:replication_ns_1@10.1.3.117:ack_seqno: 10211
eq_tapq:replication_ns_1@10.1.3.117:ack_window_full: false
eq_tapq:replication_ns_1@10.1.3.117:backfill_completed: true
eq_tapq:replication_ns_1@10.1.3.117:bg_backlog_size: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_jobs_completed: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_jobs_issued: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_queued: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_result_size: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_results: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_wait_for_results: false
eq_tapq:replication_ns_1@10.1.3.117:complete: false
eq_tapq:replication_ns_1@10.1.3.117:connected: true
eq_tapq:replication_ns_1@10.1.3.117:created: 111
eq_tapq:replication_ns_1@10.1.3.117:disconnects: 20
eq_tapq:replication_ns_1@10.1.3.117:empty: true
eq_tapq:replication_ns_1@10.1.3.117:flags: 85 (ack,backfill,vblist,checkpoints)
eq_tapq:replication_ns_1@10.1.3.117:has_item: false
eq_tapq:replication_ns_1@10.1.3.117:has_queued_item: false
eq_tapq:replication_ns_1@10.1.3.117:idle: true
eq_tapq:replication_ns_1@10.1.3.117:num_tap_nack: 0
eq_tapq:replication_ns_1@10.1.3.117:num_tap_tmpfail_survivors: 0
eq_tapq:replication_ns_1@10.1.3.117:paused: 1
eq_tapq:replication_ns_1@10.1.3.117:pending_backfill: false
eq_tapq:replication_ns_1@10.1.3.117:pending_disconnect: false
eq_tapq:replication_ns_1@10.1.3.117:pending_disk_backfill: false
eq_tapq:replication_ns_1@10.1.3.117:qlen: 0
eq_tapq:replication_ns_1@10.1.3.117:qlen_high_pri: 0
eq_tapq:replication_ns_1@10.1.3.117:qlen_low_pri: 0
eq_tapq:replication_ns_1@10.1.3.117:queue_backfillremaining: 0
eq_tapq:replication_ns_1@10.1.3.117:queue_backoff: 0
eq_tapq:replication_ns_1@10.1.3.117:queue_drain: 9839
eq_tapq:replication_ns_1@10.1.3.117:queue_fill: 0
eq_tapq:replication_ns_1@10.1.3.117:queue_itemondisk: 0
eq_tapq:replication_ns_1@10.1.3.117:queue_memory: 0
eq_tapq:replication_ns_1@10.1.3.117:rec_fetched: 10110
eq_tapq:replication_ns_1@10.1.3.117:reconnects: 20
eq_tapq:replication_ns_1@10.1.3.117:recv_ack_seqno: 10210
eq_tapq:replication_ns_1@10.1.3.117:reserved: 1
eq_tapq:replication_ns_1@10.1.3.117:seqno_ack_requested: 10210
eq_tapq:replication_ns_1@10.1.3.117:supports_ack: true
eq_tapq:replication_ns_1@10.1.3.117:suspended: false
eq_tapq:replication_ns_1@10.1.3.117:total_backlog_size: 0
eq_tapq:replication_ns_1@10.1.3.117:total_noops: 52
eq_tapq:replication_ns_1@10.1.3.117:type: producer
eq_tapq:replication_ns_1@10.1.3.117:vb_filter:

{ [33,53] }
eq_tapq:replication_ns_1@10.1.3.117:vb_filters: 21
117
ep_tap_ack_grace_period: 300
ep_tap_ack_interval: 1000
ep_tap_ack_window_size: 10
ep_tap_backoff_period: 5
ep_tap_bg_fetch_requeued: 0
ep_tap_bg_fetched: 0
ep_tap_bg_max_pending: 500
ep_tap_count: 4
ep_tap_deletes: 5710
ep_tap_fg_fetched: 75181
ep_tap_keepalive: 300
ep_tap_noop_interval: 20
ep_tap_queue_backfillremaining: 0
ep_tap_queue_backoff: 0
ep_tap_queue_drain: 45074
ep_tap_queue_fill: 0
ep_tap_queue_itemondisk: 0
ep_tap_throttle_queue_cap: 1000000
ep_tap_throttle_threshold: 90
ep_tap_throttled: 0
ep_tap_total_backlog_size: 0
ep_tap_total_fetched: 82482
ep_tap_total_queue: 0
eq_tapq:anon_696:connected: true
eq_tapq:anon_696:created: 121
eq_tapq:anon_696:num_checkpoint_end: 21
eq_tapq:anon_696:num_checkpoint_end_failed: 0
eq_tapq:anon_696:num_checkpoint_start: 22
eq_tapq:anon_696:num_checkpoint_start_failed: 0
eq_tapq:anon_696:num_delete: 127
eq_tapq:anon_696:num_delete_failed: 0
eq_tapq:anon_696:num_flush: 0
eq_tapq:anon_696:num_flush_failed: 0
eq_tapq:anon_696:num_mutation: 1277
eq_tapq:anon_696:num_mutation_failed: 0
eq_tapq:anon_696:num_opaque: 4
eq_tapq:anon_696:num_opaque_failed: 0
eq_tapq:anon_696:num_unknown: 0
eq_tapq:anon_696:num_vbucket_set: 0
eq_tapq:anon_696:num_vbucket_set_failed: 0
eq_tapq:anon_696:pending_disconnect: false
eq_tapq:anon_696:reserved: 0
eq_tapq:anon_696:supports_ack: true
eq_tapq:anon_696:type: consumer
eq_tapq:anon_705:connected: true
eq_tapq:anon_705:created: 122
eq_tapq:anon_705:num_checkpoint_end: 22
eq_tapq:anon_705:num_checkpoint_end_failed: 0
eq_tapq:anon_705:num_checkpoint_start: 23
eq_tapq:anon_705:num_checkpoint_start_failed: 0
eq_tapq:anon_705:num_delete: 98
eq_tapq:anon_705:num_delete_failed: 0
eq_tapq:anon_705:num_flush: 0
eq_tapq:anon_705:num_flush_failed: 0
eq_tapq:anon_705:num_mutation: 1086
eq_tapq:anon_705:num_mutation_failed: 0
eq_tapq:anon_705:num_opaque: 4
eq_tapq:anon_705:num_opaque_failed: 0
eq_tapq:anon_705:num_unknown: 0
eq_tapq:anon_705:num_vbucket_set: 0
eq_tapq:anon_705:num_vbucket_set_failed: 0
eq_tapq:anon_705:pending_disconnect: false
eq_tapq:anon_705:reserved: 0
eq_tapq:anon_705:supports_ack: true
eq_tapq:anon_705:type: consumer
eq_tapq:replication_ns_1@10.1.3.115:ack_log_size: 0
eq_tapq:replication_ns_1@10.1.3.115:ack_playback_size: 0
eq_tapq:replication_ns_1@10.1.3.115:ack_seqno: 35928
eq_tapq:replication_ns_1@10.1.3.115:ack_window_full: false
eq_tapq:replication_ns_1@10.1.3.115:backfill_completed: true
eq_tapq:replication_ns_1@10.1.3.115:bg_backlog_size: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_jobs_completed: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_jobs_issued: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_queued: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_result_size: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_results: 0
eq_tapq:replication_ns_1@10.1.3.115:bg_wait_for_results: false
eq_tapq:replication_ns_1@10.1.3.115:complete: false
eq_tapq:replication_ns_1@10.1.3.115:connected: true
eq_tapq:replication_ns_1@10.1.3.115:created: 62
eq_tapq:replication_ns_1@10.1.3.115:disconnects: 42
eq_tapq:replication_ns_1@10.1.3.115:empty: true
eq_tapq:replication_ns_1@10.1.3.115:flags: 85 (ack,backfill,vblist,checkpoints)
eq_tapq:replication_ns_1@10.1.3.115:has_item: false
eq_tapq:replication_ns_1@10.1.3.115:has_queued_item: false
eq_tapq:replication_ns_1@10.1.3.115:idle: true
eq_tapq:replication_ns_1@10.1.3.115:num_tap_nack: 0
eq_tapq:replication_ns_1@10.1.3.115:num_tap_tmpfail_survivors: 0
eq_tapq:replication_ns_1@10.1.3.115:paused: 1
eq_tapq:replication_ns_1@10.1.3.115:pending_backfill: false
eq_tapq:replication_ns_1@10.1.3.115:pending_disconnect: false
eq_tapq:replication_ns_1@10.1.3.115:pending_disk_backfill: false
eq_tapq:replication_ns_1@10.1.3.115:qlen: 0
eq_tapq:replication_ns_1@10.1.3.115:qlen_high_pri: 0
eq_tapq:replication_ns_1@10.1.3.115:qlen_low_pri: 0
eq_tapq:replication_ns_1@10.1.3.115:queue_backfillremaining: 0
eq_tapq:replication_ns_1@10.1.3.115:queue_backoff: 0
eq_tapq:replication_ns_1@10.1.3.115:queue_drain: 35235
eq_tapq:replication_ns_1@10.1.3.115:queue_fill: 0
eq_tapq:replication_ns_1@10.1.3.115:queue_itemondisk: 0
eq_tapq:replication_ns_1@10.1.3.115:queue_memory: 0
eq_tapq:replication_ns_1@10.1.3.115:rec_fetched: 35740
eq_tapq:replication_ns_1@10.1.3.115:reconnects: 42
eq_tapq:replication_ns_1@10.1.3.115:recv_ack_seqno: 35927
eq_tapq:replication_ns_1@10.1.3.115:reserved: 1
eq_tapq:replication_ns_1@10.1.3.115:seqno_ack_requested: 35927
eq_tapq:replication_ns_1@10.1.3.115:supports_ack: true
eq_tapq:replication_ns_1@10.1.3.115:suspended: false
eq_tapq:replication_ns_1@10.1.3.115:total_backlog_size: 0
eq_tapq:replication_ns_1@10.1.3.115:total_noops: 54
eq_tapq:replication_ns_1@10.1.3.115:type: producer
eq_tapq:replication_ns_1@10.1.3.115:vb_filter: { [11,21], [54,63] }
eq_tapq:replication_ns_1@10.1.3.115:vb_filters: 21
eq_tapq:replication_ns_1@10.1.3.117:ack_log_size: 0
eq_tapq:replication_ns_1@10.1.3.117:ack_playback_size: 0
eq_tapq:replication_ns_1@10.1.3.117:ack_seqno: 10211
eq_tapq:replication_ns_1@10.1.3.117:ack_window_full: false
eq_tapq:replication_ns_1@10.1.3.117:backfill_completed: true
eq_tapq:replication_ns_1@10.1.3.117:bg_backlog_size: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_jobs_completed: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_jobs_issued: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_queued: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_result_size: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_results: 0
eq_tapq:replication_ns_1@10.1.3.117:bg_wait_for_results: false
eq_tapq:replication_ns_1@10.1.3.117:complete: false
eq_tapq:replication_ns_1@10.1.3.117:connected: true
eq_tapq:replication_ns_1@10.1.3.117:created: 111
eq_tapq:replication_ns_1@10.1.3.117:disconnects: 20
eq_tapq:replication_ns_1@10.1.3.117:empty: true
eq_tapq:replication_ns_1@10.1.3.117:flags: 85 (ack,backfill,vblist,checkpoints)
eq_tapq:replication_ns_1@10.1.3.117:has_item: false
eq_tapq:replication_ns_1@10.1.3.117:has_queued_item: false
eq_tapq:replication_ns_1@10.1.3.117:idle: true
eq_tapq:replication_ns_1@10.1.3.117:num_tap_nack: 0
eq_tapq:replication_ns_1@10.1.3.117:num_tap_tmpfail_survivors: 0
eq_tapq:replication_ns_1@10.1.3.117:paused: 1
eq_tapq:replication_ns_1@10.1.3.117:pending_backfill: false
eq_tapq:replication_ns_1@10.1.3.117:pending_disconnect: false
eq_tapq:replication_ns_1@10.1.3.117:pending_disk_backfill: false
eq_tapq:replication_ns_1@10.1.3.117:qlen: 0
eq_tapq:replication_ns_1@10.1.3.117:qlen_high_pri: 0
eq_tapq:replication_ns_1@10.1.3.117:qlen_low_pri: 0
eq_tapq:replication_ns_1@10.1.3.117:queue_backfillremaining: 0
eq_tapq:replication_ns_1@10.1.3.117:queue_backoff: 0
eq_tapq:replication_ns_1@10.1.3.117:queue_drain: 9839
eq_tapq:replication_ns_1@10.1.3.117:queue_fill: 0
eq_tapq:replication_ns_1@10.1.3.117:queue_itemondisk: 0
eq_tapq:replication_ns_1@10.1.3.117:queue_memory: 0
eq_tapq:replication_ns_1@10.1.3.117:rec_fetched: 10110
eq_tapq:replication_ns_1@10.1.3.117:reconnects: 20
eq_tapq:replication_ns_1@10.1.3.117:recv_ack_seqno: 10210
eq_tapq:replication_ns_1@10.1.3.117:reserved: 1
eq_tapq:replication_ns_1@10.1.3.117:seqno_ack_requested: 10210
eq_tapq:replication_ns_1@10.1.3.117:supports_ack: true
eq_tapq:replication_ns_1@10.1.3.117:suspended: false
eq_tapq:replication_ns_1@10.1.3.117:total_backlog_size: 0
eq_tapq:replication_ns_1@10.1.3.117:total_noops: 52
eq_tapq:replication_ns_1@10.1.3.117:type: producer
eq_tapq:replication_ns_1@10.1.3.117:vb_filter: { [33,53] }

eq_tapq:replication_ns_1@10.1.3.117:vb_filters: 21

117
vb_0:last_closed_checkpoint_id: 1
vb_0:num_checkpoint_items: 1
vb_0:num_checkpoints: 1
vb_0:num_items_for_persistence: 0
vb_0:num_tap_cursors: 0
vb_0:open_checkpoint_id: 2
vb_0:persisted_checkpoint_id: 1
vb_0:state: replica
vb_10:last_closed_checkpoint_id: 1
vb_10:num_checkpoint_items: 1
vb_10:num_checkpoints: 1
vb_10:num_items_for_persistence: 0
vb_10:num_tap_cursors: 0
vb_10:open_checkpoint_id: 2
vb_10:persisted_checkpoint_id: 1
vb_10:state: replica
vb_11:last_closed_checkpoint_id: 1
vb_11:num_checkpoint_items: 1
vb_11:num_checkpoints: 1
vb_11:num_items_for_persistence: 0
vb_11:num_tap_cursors: 1
vb_11:open_checkpoint_id: 2
vb_11:persisted_checkpoint_id: 1
vb_11:state: replica
vb_12:last_closed_checkpoint_id: 1
vb_12:num_checkpoint_items: 1
vb_12:num_checkpoints: 1
vb_12:num_items_for_persistence: 0
vb_12:num_tap_cursors: 1
vb_12:open_checkpoint_id: 2
vb_12:persisted_checkpoint_id: 1
vb_12:state: replica
vb_13:last_closed_checkpoint_id: 1
vb_13:num_checkpoint_items: 1
vb_13:num_checkpoints: 1
vb_13:num_items_for_persistence: 0
vb_13:num_tap_cursors: 1
vb_13:open_checkpoint_id: 2
vb_13:persisted_checkpoint_id: 1
vb_13:state: replica
vb_14:last_closed_checkpoint_id: 1
vb_14:num_checkpoint_items: 1
vb_14:num_checkpoints: 1
vb_14:num_items_for_persistence: 0
vb_14:num_tap_cursors: 1
vb_14:open_checkpoint_id: 2
vb_14:persisted_checkpoint_id: 1
vb_14:state: replica
vb_15:last_closed_checkpoint_id: 1
vb_15:num_checkpoint_items: 1
vb_15:num_checkpoints: 1
vb_15:num_items_for_persistence: 0
vb_15:num_tap_cursors: 1
vb_15:open_checkpoint_id: 2
vb_15:persisted_checkpoint_id: 1
vb_15:state: replica
vb_16:last_closed_checkpoint_id: 1
vb_16:num_checkpoint_items: 1
vb_16:num_checkpoints: 1
vb_16:num_items_for_persistence: 0
vb_16:num_tap_cursors: 1
vb_16:open_checkpoint_id: 2
vb_16:persisted_checkpoint_id: 1
vb_16:state: replica
vb_17:last_closed_checkpoint_id: 1
vb_17:num_checkpoint_items: 1
vb_17:num_checkpoints: 1
vb_17:num_items_for_persistence: 0
vb_17:num_tap_cursors: 1
vb_17:open_checkpoint_id: 2
vb_17:persisted_checkpoint_id: 1
vb_17:state: replica
vb_18:last_closed_checkpoint_id: 1
vb_18:num_checkpoint_items: 1
vb_18:num_checkpoints: 1
vb_18:num_items_for_persistence: 0
vb_18:num_tap_cursors: 1
vb_18:open_checkpoint_id: 2
vb_18:persisted_checkpoint_id: 1
vb_18:state: replica
vb_19:last_closed_checkpoint_id: 1
vb_19:num_checkpoint_items: 1
vb_19:num_checkpoints: 1
vb_19:num_items_for_persistence: 0
vb_19:num_tap_cursors: 1
vb_19:open_checkpoint_id: 2
vb_19:persisted_checkpoint_id: 1
vb_19:state: replica
vb_1:last_closed_checkpoint_id: 1
vb_1:num_checkpoint_items: 1
vb_1:num_checkpoints: 1
vb_1:num_items_for_persistence: 0
vb_1:num_tap_cursors: 0
vb_1:open_checkpoint_id: 2
vb_1:persisted_checkpoint_id: 1
vb_1:state: replica
vb_20:last_closed_checkpoint_id: 1
vb_20:num_checkpoint_items: 1
vb_20:num_checkpoints: 1
vb_20:num_items_for_persistence: 0
vb_20:num_tap_cursors: 1
vb_20:open_checkpoint_id: 2
vb_20:persisted_checkpoint_id: 1
vb_20:state: replica
vb_21:last_closed_checkpoint_id: 1
vb_21:num_checkpoint_items: 1
vb_21:num_checkpoints: 1
vb_21:num_items_for_persistence: 0
vb_21:num_tap_cursors: 1
vb_21:open_checkpoint_id: 2
vb_21:persisted_checkpoint_id: 1
vb_21:state: replica
vb_22:last_closed_checkpoint_id: 1
vb_22:num_checkpoint_items: 1
vb_22:num_checkpoints: 1
vb_22:num_items_for_persistence: 0
vb_22:num_tap_cursors: 0
vb_22:open_checkpoint_id: 2
vb_22:persisted_checkpoint_id: 1
vb_22:state: replica
vb_23:last_closed_checkpoint_id: 1
vb_23:num_checkpoint_items: 1
vb_23:num_checkpoints: 1
vb_23:num_items_for_persistence: 0
vb_23:num_tap_cursors: 0
vb_23:open_checkpoint_id: 2
vb_23:persisted_checkpoint_id: 1
vb_23:state: replica
vb_24:last_closed_checkpoint_id: 1
vb_24:num_checkpoint_items: 1
vb_24:num_checkpoints: 1
vb_24:num_items_for_persistence: 0
vb_24:num_tap_cursors: 0
vb_24:open_checkpoint_id: 2
vb_24:persisted_checkpoint_id: 1
vb_24:state: replica
vb_25:last_closed_checkpoint_id: 1
vb_25:num_checkpoint_items: 1
vb_25:num_checkpoints: 1
vb_25:num_items_for_persistence: 0
vb_25:num_tap_cursors: 0
vb_25:open_checkpoint_id: 2
vb_25:persisted_checkpoint_id: 1
vb_25:state: replica
vb_26:last_closed_checkpoint_id: 1
vb_26:num_checkpoint_items: 1
vb_26:num_checkpoints: 1
vb_26:num_items_for_persistence: 0
vb_26:num_tap_cursors: 0
vb_26:open_checkpoint_id: 2
vb_26:persisted_checkpoint_id: 1
vb_26:state: replica
vb_27:last_closed_checkpoint_id: 1
vb_27:num_checkpoint_items: 1
vb_27:num_checkpoints: 1
vb_27:num_items_for_persistence: 0
vb_27:num_tap_cursors: 0
vb_27:open_checkpoint_id: 2
vb_27:persisted_checkpoint_id: 1
vb_27:state: replica
vb_28:last_closed_checkpoint_id: 1
vb_28:num_checkpoint_items: 1
vb_28:num_checkpoints: 1
vb_28:num_items_for_persistence: 0
vb_28:num_tap_cursors: 0
vb_28:open_checkpoint_id: 2
vb_28:persisted_checkpoint_id: 1
vb_28:state: replica
vb_29:last_closed_checkpoint_id: 1
vb_29:num_checkpoint_items: 1
vb_29:num_checkpoints: 1
vb_29:num_items_for_persistence: 0
vb_29:num_tap_cursors: 0
vb_29:open_checkpoint_id: 2
vb_29:persisted_checkpoint_id: 1
vb_29:state: replica
vb_2:last_closed_checkpoint_id: 1
vb_2:num_checkpoint_items: 1
vb_2:num_checkpoints: 1
vb_2:num_items_for_persistence: 0
vb_2:num_tap_cursors: 0
vb_2:open_checkpoint_id: 2
vb_2:persisted_checkpoint_id: 1
vb_2:state: replica
vb_30:last_closed_checkpoint_id: 1
vb_30:num_checkpoint_items: 1
vb_30:num_checkpoints: 1
vb_30:num_items_for_persistence: 0
vb_30:num_tap_cursors: 0
vb_30:open_checkpoint_id: 2
vb_30:persisted_checkpoint_id: 1
vb_30:state: replica
vb_31:last_closed_checkpoint_id: 1
vb_31:num_checkpoint_items: 1
vb_31:num_checkpoints: 1
vb_31:num_items_for_persistence: 0
vb_31:num_tap_cursors: 0
vb_31:open_checkpoint_id: 2
vb_31:persisted_checkpoint_id: 1
vb_31:state: replica
vb_32:last_closed_checkpoint_id: 1
vb_32:num_checkpoint_items: 1
vb_32:num_checkpoints: 1
vb_32:num_items_for_persistence: 0
vb_32:num_tap_cursors: 0
vb_32:open_checkpoint_id: 2
vb_32:persisted_checkpoint_id: 1
vb_32:state: replica
vb_33:last_closed_checkpoint_id: 1
vb_33:num_checkpoint_items: 1
vb_33:num_checkpoints: 1
vb_33:num_items_for_persistence: 0
vb_33:num_tap_cursors: 1
vb_33:open_checkpoint_id: 2
vb_33:persisted_checkpoint_id: 1
vb_33:state: replica
vb_34:last_closed_checkpoint_id: 1
vb_34:num_checkpoint_items: 1
vb_34:num_checkpoints: 1
vb_34:num_items_for_persistence: 0
vb_34:num_tap_cursors: 1
vb_34:open_checkpoint_id: 2
vb_34:persisted_checkpoint_id: 1
vb_34:state: replica
vb_35:last_closed_checkpoint_id: 1
vb_35:num_checkpoint_items: 1
vb_35:num_checkpoints: 1
vb_35:num_items_for_persistence: 0
vb_35:num_tap_cursors: 1
vb_35:open_checkpoint_id: 2
vb_35:persisted_checkpoint_id: 1
vb_35:state: replica
vb_36:last_closed_checkpoint_id: 1
vb_36:num_checkpoint_items: 1
vb_36:num_checkpoints: 1
vb_36:num_items_for_persistence: 0
vb_36:num_tap_cursors: 1
vb_36:open_checkpoint_id: 2
vb_36:persisted_checkpoint_id: 1
vb_36:state: replica
vb_37:last_closed_checkpoint_id: 1
vb_37:num_checkpoint_items: 1
vb_37:num_checkpoints: 1
vb_37:num_items_for_persistence: 0
vb_37:num_tap_cursors: 1
vb_37:open_checkpoint_id: 2
vb_37:persisted_checkpoint_id: 1
vb_37:state: replica
vb_38:last_closed_checkpoint_id: 1
vb_38:num_checkpoint_items: 1
vb_38:num_checkpoints: 1
vb_38:num_items_for_persistence: 0
vb_38:num_tap_cursors: 1
vb_38:open_checkpoint_id: 2
vb_38:persisted_checkpoint_id: 1
vb_38:state: replica
vb_39:last_closed_checkpoint_id: 1
vb_39:num_checkpoint_items: 1
vb_39:num_checkpoints: 1
vb_39:num_items_for_persistence: 0
vb_39:num_tap_cursors: 1
vb_39:open_checkpoint_id: 2
vb_39:persisted_checkpoint_id: 1
vb_39:state: replica
vb_3:last_closed_checkpoint_id: 1
vb_3:num_checkpoint_items: 1
vb_3:num_checkpoints: 1
vb_3:num_items_for_persistence: 0
vb_3:num_tap_cursors: 0
vb_3:open_checkpoint_id: 2
vb_3:persisted_checkpoint_id: 1
vb_3:state: replica
vb_40:last_closed_checkpoint_id: 1
vb_40:num_checkpoint_items: 1
vb_40:num_checkpoints: 1
vb_40:num_items_for_persistence: 0
vb_40:num_tap_cursors: 1
vb_40:open_checkpoint_id: 2
vb_40:persisted_checkpoint_id: 1
vb_40:state: replica
vb_41:last_closed_checkpoint_id: 1
vb_41:num_checkpoint_items: 1
vb_41:num_checkpoints: 1
vb_41:num_items_for_persistence: 0
vb_41:num_tap_cursors: 1
vb_41:open_checkpoint_id: 2
vb_41:persisted_checkpoint_id: 1
vb_41:state: replica
vb_42:last_closed_checkpoint_id: 1
vb_42:num_checkpoint_items: 1
vb_42:num_checkpoints: 1
vb_42:num_items_for_persistence: 0
vb_42:num_tap_cursors: 1
vb_42:open_checkpoint_id: 2
vb_42:persisted_checkpoint_id: 1
vb_42:state: replica
vb_43:last_closed_checkpoint_id: 1
vb_43:num_checkpoint_items: 1
vb_43:num_checkpoints: 1
vb_43:num_items_for_persistence: 0
vb_43:num_tap_cursors: 1
vb_43:open_checkpoint_id: 2
vb_43:persisted_checkpoint_id: 1
vb_43:state: active
vb_44:last_closed_checkpoint_id: 1
vb_44:num_checkpoint_items: 1
vb_44:num_checkpoints: 1
vb_44:num_items_for_persistence: 0
vb_44:num_tap_cursors: 1
vb_44:open_checkpoint_id: 2
vb_44:persisted_checkpoint_id: 1
vb_44:state: active
vb_45:last_closed_checkpoint_id: 1
vb_45:num_checkpoint_items: 1
vb_45:num_checkpoints: 1
vb_45:num_items_for_persistence: 0
vb_45:num_tap_cursors: 1
vb_45:open_checkpoint_id: 2
vb_45:persisted_checkpoint_id: 1
vb_45:state: active
vb_46:last_closed_checkpoint_id: 1
vb_46:num_checkpoint_items: 1
vb_46:num_checkpoints: 1
vb_46:num_items_for_persistence: 0
vb_46:num_tap_cursors: 1
vb_46:open_checkpoint_id: 2
vb_46:persisted_checkpoint_id: 1
vb_46:state: active
vb_47:last_closed_checkpoint_id: 1
vb_47:num_checkpoint_items: 1
vb_47:num_checkpoints: 1
vb_47:num_items_for_persistence: 0
vb_47:num_tap_cursors: 1
vb_47:open_checkpoint_id: 2
vb_47:persisted_checkpoint_id: 1
vb_47:state: active
vb_48:last_closed_checkpoint_id: 1
vb_48:num_checkpoint_items: 1
vb_48:num_checkpoints: 1
vb_48:num_items_for_persistence: 0
vb_48:num_tap_cursors: 1
vb_48:open_checkpoint_id: 2
vb_48:persisted_checkpoint_id: 1
vb_48:state: active
vb_49:last_closed_checkpoint_id: 1
vb_49:num_checkpoint_items: 1
vb_49:num_checkpoints: 1
vb_49:num_items_for_persistence: 0
vb_49:num_tap_cursors: 1
vb_49:open_checkpoint_id: 2
vb_49:persisted_checkpoint_id: 1
vb_49:state: active
vb_4:last_closed_checkpoint_id: 1
vb_4:num_checkpoint_items: 1
vb_4:num_checkpoints: 1
vb_4:num_items_for_persistence: 0
vb_4:num_tap_cursors: 0
vb_4:open_checkpoint_id: 2
vb_4:persisted_checkpoint_id: 1
vb_4:state: replica
vb_50:last_closed_checkpoint_id: 1
vb_50:num_checkpoint_items: 1
vb_50:num_checkpoints: 1
vb_50:num_items_for_persistence: 0
vb_50:num_tap_cursors: 1
vb_50:open_checkpoint_id: 2
vb_50:persisted_checkpoint_id: 1
vb_50:state: active
vb_51:last_closed_checkpoint_id: 1
vb_51:num_checkpoint_items: 1
vb_51:num_checkpoints: 1
vb_51:num_items_for_persistence: 0
vb_51:num_tap_cursors: 1
vb_51:open_checkpoint_id: 2
vb_51:persisted_checkpoint_id: 1
vb_51:state: active
vb_52:last_closed_checkpoint_id: 1
vb_52:num_checkpoint_items: 1
vb_52:num_checkpoints: 1
vb_52:num_items_for_persistence: 0
vb_52:num_tap_cursors: 1
vb_52:open_checkpoint_id: 2
vb_52:persisted_checkpoint_id: 1
vb_52:state: active
vb_53:last_closed_checkpoint_id: 1
vb_53:num_checkpoint_items: 1
vb_53:num_checkpoints: 1
vb_53:num_items_for_persistence: 0
vb_53:num_tap_cursors: 1
vb_53:open_checkpoint_id: 2
vb_53:persisted_checkpoint_id: 1
vb_53:state: active
vb_54:last_closed_checkpoint_id: 1
vb_54:num_checkpoint_items: 1
vb_54:num_checkpoints: 1
vb_54:num_items_for_persistence: 0
vb_54:num_tap_cursors: 1
vb_54:open_checkpoint_id: 2
vb_54:persisted_checkpoint_id: 1
vb_54:state: active
vb_55:last_closed_checkpoint_id: 1
vb_55:num_checkpoint_items: 1
vb_55:num_checkpoints: 1
vb_55:num_items_for_persistence: 0
vb_55:num_tap_cursors: 1
vb_55:open_checkpoint_id: 2
vb_55:persisted_checkpoint_id: 1
vb_55:state: active
vb_56:last_closed_checkpoint_id: 1
vb_56:num_checkpoint_items: 1
vb_56:num_checkpoints: 1
vb_56:num_items_for_persistence: 0
vb_56:num_tap_cursors: 1
vb_56:open_checkpoint_id: 2
vb_56:persisted_checkpoint_id: 1
vb_56:state: active
vb_57:last_closed_checkpoint_id: 1
vb_57:num_checkpoint_items: 1
vb_57:num_checkpoints: 1
vb_57:num_items_for_persistence: 0
vb_57:num_tap_cursors: 1
vb_57:open_checkpoint_id: 2
vb_57:persisted_checkpoint_id: 1
vb_57:state: active
vb_58:last_closed_checkpoint_id: 1
vb_58:num_checkpoint_items: 1
vb_58:num_checkpoints: 1
vb_58:num_items_for_persistence: 0
vb_58:num_tap_cursors: 1
vb_58:open_checkpoint_id: 2
vb_58:persisted_checkpoint_id: 1
vb_58:state: active
vb_59:last_closed_checkpoint_id: 1
vb_59:num_checkpoint_items: 1
vb_59:num_checkpoints: 1
vb_59:num_items_for_persistence: 0
vb_59:num_tap_cursors: 1
vb_59:open_checkpoint_id: 2
vb_59:persisted_checkpoint_id: 1
vb_59:state: active
vb_5:last_closed_checkpoint_id: 1
vb_5:num_checkpoint_items: 1
vb_5:num_checkpoints: 1
vb_5:num_items_for_persistence: 0
vb_5:num_tap_cursors: 0
vb_5:open_checkpoint_id: 2
vb_5:persisted_checkpoint_id: 1
vb_5:state: replica
vb_60:last_closed_checkpoint_id: 1
vb_60:num_checkpoint_items: 1
vb_60:num_checkpoints: 1
vb_60:num_items_for_persistence: 0
vb_60:num_tap_cursors: 1
vb_60:open_checkpoint_id: 2
vb_60:persisted_checkpoint_id: 1
vb_60:state: active
vb_61:last_closed_checkpoint_id: 1
vb_61:num_checkpoint_items: 1
vb_61:num_checkpoints: 1
vb_61:num_items_for_persistence: 0
vb_61:num_tap_cursors: 1
vb_61:open_checkpoint_id: 2
vb_61:persisted_checkpoint_id: 1
vb_61:state: active
vb_62:last_closed_checkpoint_id: 1
vb_62:num_checkpoint_items: 1
vb_62:num_checkpoints: 1
vb_62:num_items_for_persistence: 0
vb_62:num_tap_cursors: 1
vb_62:open_checkpoint_id: 2
vb_62:persisted_checkpoint_id: 1
vb_62:state: active
vb_63:last_closed_checkpoint_id: 1
vb_63:num_checkpoint_items: 1
vb_63:num_checkpoints: 1
vb_63:num_items_for_persistence: 0
vb_63:num_tap_cursors: 1
vb_63:open_checkpoint_id: 2
vb_63:persisted_checkpoint_id: 1
vb_63:state: active
vb_6:last_closed_checkpoint_id: 1
vb_6:num_checkpoint_items: 1
vb_6:num_checkpoints: 1
vb_6:num_items_for_persistence: 0
vb_6:num_tap_cursors: 0
vb_6:open_checkpoint_id: 2
vb_6:persisted_checkpoint_id: 1
vb_6:state: replica
vb_7:last_closed_checkpoint_id: 1
vb_7:num_checkpoint_items: 1
vb_7:num_checkpoints: 1
vb_7:num_items_for_persistence: 0
vb_7:num_tap_cursors: 0
vb_7:open_checkpoint_id: 2
vb_7:persisted_checkpoint_id: 1
vb_7:state: replica
vb_8:last_closed_checkpoint_id: 1
vb_8:num_checkpoint_items: 1
vb_8:num_checkpoints: 1
vb_8:num_items_for_persistence: 0
vb_8:num_tap_cursors: 0
vb_8:open_checkpoint_id: 2
vb_8:persisted_checkpoint_id: 1
vb_8:state: replica
vb_9:last_closed_checkpoint_id: 1
vb_9:num_checkpoint_items: 1
vb_9:num_checkpoints: 1
vb_9:num_items_for_persistence: 0
vb_9:num_tap_cursors: 0
vb_9:open_checkpoint_id: 2
vb_9:persisted_checkpoint_id: 1
vb_9:state: replica